search

marcom / Llama.jl

Julia interface to llama.cpp, a C/C++ library for running language models
MIT License
23 stars 2 forks source link

Add download_model #9

Closed svilupp closed 8 months ago

svilupp commented 8 months ago

This PR adds a small utility to make it super easy to download hugging face models locally. It introduces a dependency, but it's fairly standard and lightweight.

# Download the Rocket model (~1GB)
url = "https://huggingface.co/ikawrakow/various-2bit-sota-gguf/resolve/main/rocket-3b-2.76bpw.gguf"
model = download_model(url) 
# Output: "models/rocket-3b-2.76bpw.gguf"

It's quite restrictive (URL checks) to avoid common pitfalls for first-time users.

As part of the PR, I was checking tests, so I've added Aqua and its badge. Happy to remove it but it's a good practice to have it? (the movements in Project.toml are because of Aqua standards)

svilupp commented 8 months ago

Added also a mention in the README as discussed in the issue about the next steps.

We should be good to go, but I'll wait for your final nod.