search

containers / ramalama

The goal of RamaLama is to make working with AI boring.
MIT License
228 stars 34 forks source link

Replace huggingface-cli with simple client that pulls models over https #335

Open ericcurtin opened 5 hours ago

ericcurtin commented 5 hours ago

We can just pull .gguf files using a url formulated like this:

https://huggingface.co/TheBloke/Mixtral-8x7B-v0.1-GGUF/resolve/main/mixtral-8x7b-v0.1.Q4_K_M.gguf

one less dependancy needed huggingface-cli isn't packaged widely.

ericcurtin commented 5 hours ago

@swarajpande5 might be interested in taking this on