search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.37k stars 1.28k forks source link

ImportError when importing FastLanguageModel from unsloth #1313

Open MurphyJUAN opened 1 day ago

MurphyJUAN commented 1 day ago

After updating unsloth to the latest version, I encountered an ImportError when attempting to import FastLanguageModel from the package. The error message is as follows:

ImportError: cannot import name 'Replicate' from 'torch.distributed.tensor'

This error occurs when executing the following code:

from unsloth import FastLanguageModel

Environment: Python version: 3.10 PyTorch version: 2.4.1+cu121 unsloth version: unsloth-2024.11.7 CUDA version: 12.1 OS: Linux Ubuntu 20.04 Environment: Conda environment

Could you please provide guidance on how to resolve this issue or if there’s a specific version of torch that unsloth is currently compatible with?

Thank you!

zhenduow commented 16 hours ago

Hi,

I had a similar error 

ImportError: cannot import name 'patch_compiled_autograd' from 'unsloth_zoo.patching_utils'

when I ran the notebook , cell 2.

Thank you!

danielhanchen commented 7 hours ago

@zhenduow You can try updating Unsloth-zoo via pip install --upgrade unsloth-zoo

danielhanchen commented 7 hours ago

@MurphyJUAN That is a very weird error hmmm - maybe best to uninstall Unsloth entirely and reinstall it