[FEATURE] Apple Silicon: Add support of the mlx-lm

h2oai / h2o-llmstudio

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

https://h2o.ai

Apache License 2.0

4.03k stars 418 forks source link

Open qdrddr opened 5 months ago

qdrddr commented 5 months ago

Hi, can you please consider adding Apple Silicon support? mlx-lm makes it very easy to fine-tune LLM. That'll be great having UI for this framework from H2O LLM Studio. pip install mlx-lm torch sentencepiece https://medium.com/@anchen.li/fine-tune-llama3-with-function-calling-via-mlx-lm-5ebbee41558f

Utilize powerful M-Ultra GPUs

qdrddr commented 5 months ago

To answer why:

Powerful macs: MaxStudio or Mac Pro with M2 Ultra and 196GB RAM available for GPU with 60 cores.
Apple is developing Apple Silicon Project ACDC designed specifically for server farms dedicated to AI processing.