The issue Multiple dispatch failed for 'torch._ops.aten.to.dtype_layout' is relevant again

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

https://unsloth.ai

Apache License 2.0

16.7k stars 1.15k forks source link

The issue Multiple dispatch failed for 'torch._ops.aten.to.dtype_layout' is relevant again #1126

Open yurkoff-mv opened 12 hours ago

yurkoff-mv commented 12 hours ago

Issue: https://github.com/unslothai/unsloth/issues/741#top

package                   Version
------------------------- --------------
accelerate                0.34.2
bitsandbytes              0.44.1
datasets                  2.21.0
flash-attn                2.6.3
numpy                     1.26.4
nvgpu                     0.10.0
peft                      0.13.0
pynvml                    11.5.3
safetensors               0.4.5
sentencepiece             0.2.0
tokenizers                0.20.0
torch                     2.3.1+cu121
torchaudio                2.3.1+cu121
torchtext                 0.18.0
torchvision               0.18.1+cu121
transformers              4.45.1
trl                       0.11.1
unsloth                   2024.9.post4
xformers                  0.0.27

yurkoff-mv commented 10 hours ago

The following command solves the problem, but conflicts with dependencies: pip install --no-deps "xformers<0.0.27"