search

mzbac / mlx-moe

Scripts to create your own moe models using mlx
86 stars 6 forks source link

Can this project run on non-Apple chips? #3

Open blackblue9 opened 8 months ago

blackblue9 commented 8 months ago

Can this project run on non-Apple chips? My environment is NVIDIA's A800 because I saw that the mlx library used by the project is designed for Apple chips and systems.

I can get the moe model file after running the moe.py code, but problems occur when running inference.py and lora.py, such as when running the lora code:

` python lora.py

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Total parameters 16389.249M Trainable parameters 10.487M Starting training..., iters: 9000 Aborted ` It stopped for no reason, I don’t know what the problem is.

mzbac commented 8 months ago

I don't think the MLX supports fine-tuning on CUDA. However, since you already have the MOE model which is compatible with Transformers, you can directly fine-tune it using the Hugging Face TRL library. FYI: https://huggingface.co/docs/trl/en/sft_trainer#quickstart