search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
17.88k stars 1.24k forks source link

Global Notebook Support #301

Open nickthome opened 7 months ago

nickthome commented 7 months ago

Currently cannot run the mistral example notebook on Databricks despite matching up package specifications on the underlying image. The runs just kill the python kernel and force a restart of the cluster once in trainer.train(), and the same occurs on our jupyterhub instance. Seems like the way the underlying image is interacted with on Colab is different than the interactions elsewhere.

No idea where to being to solve this, but being able to use this package on Databricks or other notebook uis would be a huge boon and business opportunity for you.

danielhanchen commented 7 months ago

Hmm good point - on Databricks VMs - someone in our Discord chat said using ddp_find_unused_parameters=False in TrainingArguments made 1x A10G instance work - unsure if that's the solution