Add 4-bit MLX models of small models

transformerlab / transformerlab-app

Open Source Application for Advanced LLM Engineering: interact, train, fine-tune, and evaluate large language models on your own computer.

https://transformerlab.ai/

GNU Affero General Public License v3.0

424 stars 24 forks source link

Open dadmobile opened 4 months ago

dadmobile commented 4 months ago

Maybe some of TinyLlama, Phi, Qwen2 small models?

delfireinoso commented 4 months ago

The light models have a lot of uses, not only for underpowered macs. On MOA situations light models can solve a lot of problems

dadmobile commented 3 months ago

True! Just copied this over from a request. Updated title.