Try Code Llama

[simonw]

See https://ai.meta.com/blog/code-llama-large-language-model-coding/

From https://huggingface.co/TheBloke/CodeLlama-13B-Python-GGML/tree/main

llm llama-cpp download-model \
  https://huggingface.co/TheBloke/CodeLlama-13B-Python-GGML/resolve/main/codellama-13b-python.ggmlv3.Q4_K_S.bin \
  --alias code13b

It's a 7.37GB download.

[simonw]

Not great on first attempt:

---

2023-08-24T22:48:50 conversation: 01h8mvsfn1n5a9gnr7vd3h48tb

Model: codellama-13b-python.ggmlv3.Q4_K_S

Prompt:

python code to download a CSV from a URL and create a SQLite table for it and write the rows to the table

Response:

.

import sqlite3
from urllib import request as httpRequest # Python 3

# get a CSV from an internet URL via HTTP/S GET requests; load that CSV into a in-memory ( SQLite database, and write the loaded data into a SQLite table in that SQLite database.

def main():

    sqlDatabaseFilePath = "./database.sqlite"

    # create a new SQLite database file at "../database/database.sqlite", or use the already existing previously existing SQLite database file at "./database.sqlite":

    # #

[simonw]

My hunch is that there's other stuff needed here, maybe a prompt template with [INSTR] or similar in it.

[yellinben]

Could it possibly be related to this disclaimer from the intro post:

We recommend using Code Llama - Instruct variants whenever using Code Llama for code generation since Code Llama - Instruct has been fine-tuned to generate helpful and safe answers in natural language.

We do not recommend using Code Llama or Code Llama - Python to perform general natural language tasks since neither of these models are designed to follow natural language instructions.

I tried the same prompt using ollama's default codellama model (7b-Instruct), and got a much more coherent, code-filled response:

import csv
import sqlite3
with open('data.csv') as csvfile:
    reader = csv.reader(csvfile)
for row in reader:
    # Do something with each row here, e.g.:
    print(row)
conn = sqlite3.connect('mydatabase.db')
c = conn.cursor()
# Create a table for the data
c.execute("CREATE TABLE mytable (id INTEGER PRIMARY KEY, name TEXT, age INTEGER)")
# Insert rows into the table
for row in reader:
    c.execute("INSERT INTO mytable VALUES(?, ?, ?)", (row[0], row[1], row[2])))
conn.commit()
conn.close()

[bvanslyke]

Code Llama Instruct works with the llama2-chat prompt format. I just had to do:

llm llama-cpp add-model ~/llamas/codellama-34b-instruct.Q4_K_M.gguf --alias codellama --llama2-chat