OoriData / OgbujiPT

Client-side toolkit for using large language models, including where self-hosted
Apache License 2.0
102 stars 8 forks source link

Add support for download-model.py to retrieve PyTorch files (when there are both those and safetensors) #62

Open chimezie opened 11 months ago

chimezie commented 11 months ago

Some systems (such as mlx) don't yet work with HF's safetensors ( see LLama example doesn't work with HF format models? #65 ) and require access to the PyTorch files to work with and convert from them. Currently, I have to download these manually, and it would be beneficial to enhance this script to support getting just the PyTorch files, even if there are both (with an appropriate warning about the dangers of doing so, of course).

Given the dichotomy between mlx and HF models and the scarcity of current support on Apple Metal, there is general value in utilities that help bridge the gap, and this is just one example.