search

unslothai / unsloth

Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
12.85k stars 837 forks source link

Windows 11+Conda: What is right version of triton to be used to work with unsloth #460

Open sumitshrestha opened 2 months ago

sumitshrestha commented 2 months ago

I am trying to run fine tuning on llama 3 just like others here using unsloth on my local windows 11. I have set up two virtual env one using 3.10 and other using 3.11. when i installed the packages conda install pytorch-cuda=<12.1/11.8> pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers i get error. so, i installed pytorch using command provided in the official website. when i ran the python script it failed at from unsloth import FastLanguageModel the error is

ModuleNotFoundError: No module named 'triton'

So, i tried installing triton using different ways but failed. This morning i downloaded the whl file and installed triton that way. here is the link to it https://pypi.org/project/triton-library/#files. When i installed i got another error related again to triton

ModuleNotFoundError: No module named 'triton.common'

So, what would be right version of triton i should be using with unsloth?

Primary-Ad2848 commented 2 months ago

Same problem...

LiZhenzhuBlog commented 1 month ago

Same problem...

Jiar commented 1 month ago

I also have the need to use unsloth in windows, so I made a docker images, you are also welcome to use. https://github.com/Jiar/jupyter4unsloth

danielhanchen commented 1 month ago

Apologies this slipped by me! Extreme sorry! Ye unfortunately Windows is a bit of an issue to support - (due to Triton). See https://github.com/unslothai/unsloth/issues/210 which might be helpful

danielhanchen commented 1 month ago

Very cool @Jiar !! Will check that out!