search

unslothai / unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
15.59k stars 1.05k forks source link

Apple Mac Book MLX support #68

Open ehartford opened 8 months ago

ehartford commented 8 months ago

Can it work on MacBook with mlx?

danielhanchen commented 8 months ago

@ehartford Currently no sadly - but since MLX was released, if I have some time - I'll make it work for Apple devices! MLX looks pretty cool so worth a shot :)

ehartford commented 8 months ago

I'll try 😁 (but don't count on me to succeed!)

danielhanchen commented 8 months ago

Cool! We can maybe collaborate to make it happen :)

gardner commented 8 months ago

Have you two started a branch?

danielhanchen commented 8 months ago

@gardner I'm currently working on conversion to GGUF and other stuff - not sure if I can even get MLX up and running since I don't even have a Mac!!!

fblissjr commented 6 months ago

Big fan of unsloth on my CUDA PC. Thought I'd see if there's any renewed interest here for MLX?

Curious if anyone's been able to do a quick analysis on how big the effort is.

danielhanchen commented 6 months ago

@fblissjr Yep https://github.com/unslothai/unsloth/issues/4 :) So it's doable, just technically I do not have a Mac!!! Am a Windows + Linux user (shockingly!!) so I thought just adding Unsloth code to MLX could work, but then after thinking about it more, testing it will be complex - so until I get a Mac, maybe I'll work on it!!

fblissjr commented 6 months ago

@fblissjr Yep #4 :) So it's doable, just technically I do not have a Mac!!! Am a Windows + Linux user (shockingly!!) so I thought just adding Unsloth code to MLX could work, but then after thinking about it more, testing it will be complex - so until I get a Mac, maybe I'll work on it!!

If I can help, let me know. Still learning the ins and outs of MLX, but it's fairly intuitive. Not as familiar with how Unsloth works though.

jesusoctavioas commented 3 months ago

@fblissjr Yep #4 :) So it's doable, just technically I do not have a Mac!!! Am a Windows + Linux user (shockingly!!) so I thought just adding Unsloth code to MLX could work, but then after thinking about it more, testing it will be complex - so until I get a Mac, maybe I'll work on it!!

I do have a macbook pro m1 max with 64 gb of ram and some minis 2012 and a mini m2, so maybe i can help with the testing?