Account for the fact that there non-llama.cpp gguf files now

huggingface / huggingface.js

Utilities to use the Hugging Face Hub API

https://hf.co/docs/huggingface.js

MIT License

1.37k stars 213 forks source link

Account for the fact that there non-llama.cpp gguf files now #856

Closed julien-c closed 1 month ago

julien-c commented 1 month ago

companion internal PR

TL;DR: we want the ability to discriminate non-llama.cpp gguf repos

I've picked ${architecture}.context_length as a property that, if it's in the gguf file, we assume it's compatible with llama.cpp

Another option was to check whether architecture is inside the list in @huggingface/gguf (and keep this list up to date with upstream, maybe through some CI)

Both options are possible, so let me know what you think is best

julien-c commented 1 month ago

(since we'll need to keep the architecture list fully synced at all times - which gets updated quite a bit)

Yes, but IMO in the future we (e.g. @ngxson 😁) can automate the syncing of our architecture list with upstream through some CI