Meta Open Sources Code Llama 70B

CodeL lama 70B, the latest and most powerful iteration of our open-source language model for code generation. That’s right, we’re not only pushing the boundaries of AI-powered coding, but making it freely accessible. With improved performance over previous iterations, Code Llama 70B is now available under the same permissive license as prior Code Llama releases.

Notably, Code Llama 70B achieves over 67.8% on the HumanEval benchmark, reaching performance on par with GPT-4.

This isn’t just a bigger engine under the hood – it’s a leap forward in code-wrangling capabilities. Code Llama 70B boasts significant performance improvements on key benchmarks, meaning you can say goodbye to tedious boilerplate and hello to lightning-fast generation, smarter autocompletion, and even tackling diverse programming languages.

Code LLama 70B comes in three distinct versions:

• CodeLlama-70B: Your all-around powerhouse for general code generation across multiple languages.
• CodeLlama-70B-Python: Tailored for the Python-specific tasks.
• CodeLlama-70B-Instruct: Fine-tuned instruct version.

So, ready to unleash the Code Llama 70B in your projects? Buckle up, grab your access key, and prepare to experience the future of coding, where the only limit is your imagination. Dive deeper in the following sections to explore the model’s capabilities, access instructions, and see how Code Llama 70B can turbocharge your workflow. The future of code is open, and it’s here to stay.

Get ready to code smarter, not harder, with Code Llama 70B.

Decoding Code Llama 70B: Under the Hood

Let’s examine the engine driving Code Llama 70B. This section dives into the technical details, giving you a peek at the brains behind the magic.

Model Core:

• Parameter Powerhouse: This version boasts a whopping 70 billion parameters, allowing it to process and generate complex code structures with stunning accuracy. Reportedly trained on 500B tokens.
• Input/Output Dance: It takes plain text as input, weaving its insights into eloquent lines of code as output. Think of it as your AI translator for turning natural language into programming magic. Code llama 70B has a 16384 context window.
• Transformer Tango: The model’s architecture leans on the optimized transformer approach, a well-established technique in the LLM world known for its flexibility and power.

Fine-Tuning for Focus:

• Scale Matters: Code Llama 70B was fine-tuned on massive datasets with up to 16,000 tokens, ensuring it’s familiar with diverse coding structures and styles.
• Supersized Inference: During inference, it can handle even larger chunks of code, up to 100,000 tokens, making it ideal for tackling complex projects and intricate problems.
• Model Dates Code Llama and its variants have been trained between January 2023 and January 2024.

A Look Ahead:

• Static Stability: This is a static model, trained on a carefully curated dataset. This ensures consistency and predictability in its outputs.
• Community-Driven Evolution: Future versions of Code Llama – Instruct will focus on enhancing safety and refining its abilities. We’ll be actively incorporating community feedback into this process, so stay tuned for even more exciting developments!

Testing the model

Thanks to Perplexity, we can test the open source model through the Perplexity Labs feature. Here I wanted to ask Code Llama a random hard Leetcode question, one that hopefully is not in its trainnig data. I asked it question 2179. Count Good Triplets in an Array. Here is the prompt I gave it:

You are given two 0-indexed arrays nums1 and nums2 of length n, both of which are permutations of [0, 1, ..., n - 1].

A good triplet is a set of 3 distinct values which are present in increasing order by position both in nums1 and nums2. In other words, if we consider pos1v as the index of the value v in nums1 and pos2v as the index of the value v in nums2, then a good triplet will be a set (x, y, z) where 0 <= x, y, z <= n - 1, such that pos1x < pos1y < pos1z and pos2x < pos2y < pos2z.

class Solution:
    def goodTriplets(self, nums1: List[int], nums2: List[int]) -> int:

Notice I gave the prompt the main part of the question along with the start code, I did not mention the Leetcode number or question.

This the code result:

class Solution:
    def goodTriplets(self, nums1: List[int], nums2: List[int]) -> int:
        n = len(nums1)
        count = 0
        for i in range(n):
            for j in range(i + 1, n):
                for k in range(j + 1, n):
                    if nums1[i] < nums1[j] < nums1[k] and nums2[i] < nums2[j] < nums2[k]:
                        count += 1
        return count

The code compiled, but it did not pass all the tests.. I then gave it a follow up question, “try again, think step by step”. It still gave the wrong answer. I then gave the same prompt to Mistral Medium and returned this.

class Solution:
    def goodTriplets(self, nums1: List[int], nums2: List[int]) -> int:
        n = len(nums1)
        # Create two arrays to store the position of each element in nums1 and nums2
        pos1 = [0] * n
        pos2 = [0] * n
        for i in range(n):
            pos1[nums1[i]] = i
            pos2[nums2[i]] = i

        # Initialize the count of good triplets to 0
        count = 0
        # Iterate through all possible triplets
        for i in range(n):
            for j in range(i+1, n):
                for k in range(j+1, n):
                    # Check if the triplet is good
                    if pos1[i] < pos1[j] < pos1[k] and pos2[i] < pos2[j] < pos2[k]:
                        count += 1
        return count

Still, neither got the question right,, maybe with better prompting or showing examples might have helped. But let’s also note that Leetcode isn’t the best way to determine a language model’s coding abilities.

Code LLama 70B licensing

Code Llama 70B is free and open source as well as available for commercial use.