simonw / llm-llama-cpp

LLM plugin for running models using llama.cpp
Apache License 2.0
136 stars 19 forks source link

Download model should use llm.user_path() and register once complete #4

Closed simonw closed 1 year ago

simonw commented 1 year ago

That command is incomplete - it should download to llm.user_path() / "llama-cpp" / "models" and register the model in the JSON file once the download finishes.

Split from:

simonw commented 1 year ago

This worked:

llm llama-cpp download-model \
  https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q8_0.bin \
  --alias llama2-chat --alias l2c

Full session:

$ llm llama-cpp download-model \
  https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q8_0.bin \
  --alias llama2-chat --alias l2c
Downloading 6.67 GB  [####################################]  100%          
Downloaded model to /Users/simon/Library/Application Support/io.datasette.llm/llama-cpp/models/llama-2-7b-chat.ggmlv3.q8_0.bin
$ llm llama-cpp models
{
  "llama-2-7b-chat.ggmlv3.q8_0": {
    "path": "/Users/simon/Library/Application Support/io.datasette.llm/llama-cpp/models/llama-2-7b-chat.ggmlv3.q8_0.bin",
    "aliases": [
      "llama2-chat",
      "l2c"
    ]
  }
}
$ llm -m l2c 'Five names for a puppy'
 that starts with the letter 'P'
A puppy that starts with the letter "P" can have many wonderful and unique names. Here are five suggestions:
1. Paisley - This name is perfect for a playful and curious puppy. It has a fun and energetic vibe to it, and it's easy to pronounce.
2. Parker - For a puppy that's always on the go, Parker is a great choice. It's short and simple, yet it's strong and masculine.
3. Pixie - This name is perfect for a little ball of fluff that's full of energy and spirit. It's a playful and whimsical name that suits a puppy with a mischievous personality.
4. Poppet - This name is sweet and charming, and it's perfect for a small but mighty puppy. It's a fun and unique name that will make your puppy stand out from the crowd.
5. Phoenix - For a puppy that's born from ashes, Phoenix is a powerful and symbolic name. It represents new beginnings and transformation, making it perfect for a puppy that's full of energy and life.

Still need to figure this out though: