search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.56k stars 1.3k forks source link

Errors occurring in Pip Installation : torch 2.5 and CUDA 12.4 #1247

Closed daegonYu closed 2 weeks ago

daegonYu commented 2 weeks ago

Since I have cuda 12.4 version, I downloaded torch 2.5.1 version that matches it. And I ran the command below as instructed, but I was able to find the error log

pip install --upgrade pip
pip install "unsloth[cu124-torch250] @ git+https://github.com/unslothai/unsloth.git"

error log

Collecting unsloth@ git+https://github.com/unslothai/unsloth.git (from unsloth[cu124-torch250]@ git+https://github.com/unslothai/unsloth.git)
  Cloning https://github.com/unslothai/unsloth.git to /tmp/pip-install-w9iv0uc0/unsloth_b9fd73389fe14849ad82242d20ed456f
  Running command git clone --filter=blob:none --quiet https://github.com/unslothai/unsloth.git /tmp/pip-install-w9iv0uc0/unsloth_b9fd73389fe14849ad82242d20ed456f
  Resolved https://github.com/unslothai/unsloth.git to commit e2e406eba603eb605d622f17fa33660bbf4ad649
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting bitsandbytes>=0.43.3 (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-torch250]@ git+https://github.com/unslothai/unsloth.git)
  Using cached bitsandbytes-0.44.1-py3-none-manylinux_2_24_x86_64.whl.metadata (3.5 kB)
Requirement already satisfied: torch in ./anaconda3/envs/unsloth/lib/python3.10/site-packages (from bitsandbytes>=0.43.3->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-torch250]@ git+https://github.com/unslothai/unsloth.git) (2.5.1)
Requirement already satisfied: numpy in ./anaconda3/envs/unsloth/lib/python3.10/site-packages (from bitsandbytes>=0.43.3->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-torch250]@ git+https://github.com/unslothai/unsloth.git) (2.1.3)
Collecting xformers@ https://download.pytorch.org/whl/cu124/xformers-0.0.28.post2-cp310-cp310-manylinux_2_28_x86_64.whl (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-torch250]@ git+https://github.com/unslothai/unsloth.git)
  Using cached https://download.pytorch.org/whl/cu124/xformers-0.0.28.post2-cp310-cp310-manylinux_2_28_x86_64.whl (20.6 MB)
INFO: pip is looking at multiple versions of unsloth[huggingface] to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement unsloth-zoo>=2024.11.1; extra == "huggingface" (from unsloth[huggingface]) (from versions: 2024.10.0, 2024.10.1, 2024.10.2, 2024.10.3, 2024.10.4, 2024.10.5, 2024.11.0)
ERROR: No matching distribution found for unsloth-zoo>=2024.11.1; extra == "huggingface"
daegonYu commented 2 weeks ago

The same error occurred not only with the command pip install "unsloth[cu124-torch250] @ git+https://github.com/unslothai/unsloth.git" but also with the installation command pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git".

I solved it with the installation command below. It was hard to get out of that error without downloading unsloth_zoo first.

conda create --name unsloth_env \
    python=3.10 \
    pytorch-cuda=12.1 \
    pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers \
    -y
conda activate unsloth_env

# First, download unsloth_zoo
pip install "unsloth_zoo @ git+https://github.com/unslothai/unsloth-zoo.git"

pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"