search

ToluClassics / mlx-transformers

MLX Transformers is a library that provides model implementation in MLX. It uses a similar model interface as HuggingFace Transformers and provides a way to load and run models in Apple Silicon devices.
Apache License 2.0
52 stars 4 forks source link

Useful Models To add to MLX Transformers #10

Open ToluClassics opened 7 months ago

ToluClassics commented 7 months ago

Creating this issue to keep track of Models that will be nice to have ported:

--> Fuyu : https://huggingface.co/adept/fuyu-8b --> OpenELM --> CLIP --> Whisper --> SeamlessMT: facebook/seamless-m4t-medium --> TROCR: microsoft/trocr-large-handwritten --> LLAVA --> T5 --> Mistral --> Idefics: https://huggingface.co/HuggingFaceM4/idefics-9b-instruct/tree/main

ToluClassics commented 7 months ago

Fuyu: https://github.com/ToluClassics/mlx-transformers/pull/15

BuildBackBuehler commented 7 months ago

Man, I can't believe this wasn't done by the MLX team months ago! Thank you for this, should come in handy. I was going to suggest Mixtral as #1 but then I found this and screamed like a little girl

https://github.com/Goekdeniz-Guelmez/Mixtral-from-scratch-in-MLX

Seems pretty thorough -- sooo if ya see this...thanks @Goekdeniz!

Also donno if there are any tweaks necessary for Llama-3/Mixtral 8x22B but I'd throw those in too. I'll be finding out in a little bit I suppose...and if I catch that/fix it I'll put in a PR

ToluClassics commented 7 months ago

I think Mixtral would be a useful addition for sure!

ToluClassics commented 7 months ago

OpenELM: https://github.com/ToluClassics/mlx-transformers/pull/17