Add 4-bit MLX models of small models

transformerlab / transformerlab-app

Open Source Application for Advanced LLM Engineering: interact, train, fine-tune, and evaluate large language models on your own computer.

https://transformerlab.ai/

GNU Affero General Public License v3.0

280 stars 14 forks source link

Open dadmobile opened 1 month ago

dadmobile commented 1 month ago

Maybe some of TinyLlama, Phi, Qwen2 small models?

delfireinoso commented 1 month ago

The light models have a lot of uses, not only for underpowered macs. On MOA situations light models can solve a lot of problems

dadmobile commented 1 month ago

True! Just copied this over from a request. Updated title.