Phind AI Releases Their 70B Model

Phind AI has released their Phind-70B model, which is their largest and most performant model to date. This model is based on the CodeLlama-70B model and is fine-tuned on an additional 50 billion tokens, resulting in significant improvements. Phind-70B supports a context window of 32K tokens and scores 82.3% on HumanEval, surpassing the latest GPT-4 Turbo score of 81.1% in their evaluation. It also runs at up to 80 tokens per second, providing high-quality answers for technical topics efficiently.

Phind-70B is faster than GPT-4 Turbo, running at 80+ tokens per second compared to GPT-4 Turbo’s ~20 tokens per second. This speed is achieved by running NVIDIA’s TensorRT-LLM library on H100 GPUs, with ongoing optimizations to further enhance inference speed. Phind-70B is available for free to try without a login, with the option to access higher limits by subscribing to Phind Pro.

Phind-70B Examples

Solving Hard Leetcode Problems

You are given an m x n integer array grid where grid[i][j] could be:

    1 representing the starting square. There is exactly one starting square.
    2 representing the ending square. There is exactly one ending square.
    0 representing empty squares we can walk over.
    -1 representing obstacles that we cannot walk over.

Return the number of 4-directional walks from the starting square to the ending square, that walk over every non-obstacle square exactly once.

This is from Leetcode 980. Unique Paths III.

Phind-70B’s answer

from typing import List

class Solution:
    def uniquePathsIII(self, grid: List[List[int]]) -> int:
        self.directions = [(0,  1), (0, -1), (1,  0), (-1,  0)]  # Right, Left, Down, Up
        self.empty_squares =  1  # Start counting from  1 to include the start square
        self.paths =  0
        self.start = None

        # Find the starting point and count empty squares
        for i in range(len(grid)):
            for j in range(len(grid[0])):
                if grid[i][j] ==  1:
                    self.start = (i, j)
                elif grid[i][j] ==  0:
                    self.empty_squares +=  1

        def dfs(x, y, empty_squares):
            if not (0 <= x < len(grid)) or not (0 <= y < len(grid[0])) or grid[x][y] <  0:
                return  0
            if grid[x][y] ==  2:
                return  1 if empty_squares ==  0 else  0
            grid[x][y] = -2  # Temporarily mark as visited
            self.paths += sum(dfs(x + dx, y + dy, empty_squares -  1) for dx, dy in self.directions)
            grid[x][y] =  0  # Unmark after exploring all paths
            return  0

        dfs(*self.start, self.empty_squares)
        return self.paths

Unfortunately it was not able to get the initial test to pass.

Frontend UI Code

Prompt:

Generate a header using tailwind css. We want a logo on the left, a home, about and contact link in the center. On the right we want a Log In and Sign Up buttons

<header class="bg-gray-800 text-white p-4 flex items-center justify-between">
  <!-- Logo -->
  <div class="flex items-center">
    <img src="path/to/your/logo.png" alt="Logo" class="h-8 w-auto">
  </div>

  <!-- Navigation Links -->
  <nav class="hidden md:flex space-x-4">
    <a href="#" class="hover:text-gray-300">Home</a>
    <a href="#" class="hover:text-gray-300">About</a>
    <a href="#" class="hover:text-gray-300">Contact</a>
  </nav>

  <!-- Log In and Sign Up Buttons -->
  <div class="flex space-x-4">
    <button class="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded">
      Log In
    </button>
    <button class="bg-green-500 hover:bg-green-700 text-white font-bold py-2 px-4 rounded">
      Sign Up
    </button>
  </div>
</header>

Ok this one was actually pretty impressive and had great coherence.

Working With Libraries

Prompt:

Given a NumPy array arr with shape (3, 4, 5), reshape it into a new array with shape (6, 10).

import numpy as np

# Assuming arr is your original array with shape (3,  4,  5)
# arr = np.random.rand(3,  4,  5)  # Example of how to create such an array

# Reshape the array
new_arr = arr.reshape(6,  10)

print(new_arr.shape)  # This should output: (6,  10)

Explanation:

1. arr.reshape(2, 3, 4, 5): This reshapes the array arr into a four-dimensional array with shape (2, 3, 4, 5). This is an intermediate step to achieve the desired shape of (6, 10).
2. arr.transpose(0, 2, 1, 3): This transposes the axes of the array arr in the following order: (0, 2, 1, 3). This effectively swaps the first and second dimensions, and the third and fourth dimensions. This results in an array with shape (2, 4, 3, 5).
3. arr.reshape(6, 10): Finally, this reshapes the array arr into a two-dimensional array with shape (6, 10). This achieves the desired output shape.

This solution demonstrates an understanding of NumPy’s reshape and transpose functions, as well as the concept of reshaping multidimensional arrays.

Closing Thoughts

We’ve all seen the benchmarks on how Phind AI can beat GPT-4. Benchmarks tend to not be so accurate, many models seems to be contaminated just to approach proprietary models in coding benchmarks. Overall it’s a decent model, I wouldn’t say it’s better than GPT-4, but certainly better at than most, for programming at least. Also they don’t mention what it actually beats GPT on. Every time I see one of these it “wins” on incredibly simple and small functions, while not even being able to understand more complex requests.