search

exo-explore / exo

Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚
GNU General Public License v3.0
10.19k stars 568 forks source link

Support for Reflection 70B model #205

Open stephanj opened 1 month ago

stephanj commented 1 month ago

Would be great to have Reflection 70B model support, please :) https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B

AlexCheema commented 1 month ago

If someone could please convert/upload the weights then this would be supported out of the box. I've tried but my internet speed is too slow right now to do this.

The command to run inside the llms directory in this repo https://github.com/ml-explore/mlx-examples would be python3 -m mlx_lm.convert --hf-path mattshumer/Reflection-70B -q --mlx-path Reflection-70B-4bit --upload-repo mlx-community/Reflection-70B-4bit