Conda environment setup issues

[timlai4]

I am attempting to follow the new Anaconda installation setup instructions for the unsloth_env, but I am experiencing several issues. The first issue was with the PyTorch and the CUDA toolkit installation, which was an easy fix, so I will describe it and the solution last. The second issue is with the triton environment, which I describe first because I have not been able to resolve it prior to posting this.

First, following the instructions, I attempt to run from unsloth import FastLanguageModel, but I get a ModuleNotFoundError: No module named 'triton' error. Fine, I go back and install the module via conda install -c conda-forge triton. Rerunning the same command now yields another ModuleNotFoundError.

mambaforge-pypy3/envs/unsloth/lib/python3.12/site-packages/unsloth/__init__.py", line 104, in <module>
    else: from triton.common.build import libcuda_dirs
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'triton.common'

For the CUDA toolkit issue, I have CUDA 12.1 on my system, verified via nvidia-smi. However, after running

conda create --name unsloth_env \
    python=3.11 \
    pytorch-cuda=12.1 \
    pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers \
    -y

I noticed that torch.cuda.is_available() returns False. I was able to fix this issue by instead running

conda create --name unsloth pytorch torchvision torchaudio pytorch-cuda=12.1 xformers -c pytorch -c nvidia -c xformers  -y

[danielhanchen]

Hmm is this on a Windows machine?

[YaBoyBigPat]

I'm also having trouble with triton my installation is on windows using conda, I tried it using ubuntu on wsl and still ran into the same triton errors. Really I've run into a cave of errors trying to get this to work on windows, but here's my error:

---

ModuleNotFoundError Traceback (most recent call last) Cell In[1], line 1 ----> 1 from unsloth import FastLanguageModel

File c:\Users\ljjx\anaconda3\envs\unsloth_env\lib\site-packages\unsloth__init.py:103 100 # Try loading bitsandbytes and triton [101](file:///C:/Users/ljjx/anaconda3/envs/unsloth_env/lib/site-packages/unsloth/init.py:101) import bitsandbytes as bnb --> 103 import triton [104](file:///C:/Users/ljjx/anaconda3/envs/unsloth_env/lib/site-packages/unsloth/init.py:104) libcuda_dirs = lambda: None [105](file:///C:/Users/ljjx/anaconda3/envs/unsloth_env/lib/site-packages/unsloth/init.py:105) if Version(triton.version__) >= Version("3.0.0"):

ModuleNotFoundError: No module named 'triton'

When I tried pip installing it I got these errors: pip install triton ERROR: Could not find a version that satisfies the requirement triton (from versions: none) ERROR: No matching distribution found for triton

[timlai4]

Hmm is this on a Windows machine?

No, on Ubuntu 18.04.6 LTS

[mbalogun-thub]

For running on windows I believe this GitHub issue specifies how to carry out the installation. https://github.com/unslothai/unsloth/issues/210

[mbalogun-thub]

While for ubuntu not tried this on a ubuntu machine but I was having the same issue with google collab and updating the install commands as mentioned in this issue https://github.com/unslothai/unsloth/issues/501 seemed to work. You can also try the solution in this issue https://github.com/unslothai/unsloth/issues/440.

Pasting the commands below as well:

For issue 510, removing the version requirements as suggested solved for collab. However, I still had to install triton.

!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps xformers trl peft accelerate bitsandbytes
!pip install triton

For issue 440 the solution was to run: conda install gcc_linux-64

[timlai4]

This seems to have been fixed with one of the more recent changes. I remade the environment and no longer encountered the error. However, the conda installation instructions in the README still were not working for me. Instead, I had to use the one I suggested in my original post:

conda create --name unsloth pytorch torchvision torchaudio pytorch-cuda=12.1 xformers -c pytorch -c nvidia -c xformers  -y

I'm not sure if I'm the only one with this issue. Moreover, this is in line with the PyTorch installation instructions on the official website.

[YaBoyBigPat]

Nice! I'll try it out and let you know if it works for me too. Thanks man.