search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
17.73k stars 1.23k forks source link

subprocess.CalledProcessError #218

Open pauli31 opened 8 months ago

pauli31 commented 8 months ago

Hi, I'm getting the following error when I try to run unsloth on Windows with WSL, any idea what causes it? Python 3.10

model, tokenizer = FastLanguageModel.from_pretrained(mistralai/Mistral-7B-Instruct-v0.2)
FastLanguageModel.for_inference(model)
 output_ids = model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            generation_config=generation_config
        )
   output_ids = model.generate(
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
      return func(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/transformers/generation/utils.py", line 1544, in generate
      return self.greedy_search(
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/transformers/generation/utils.py", line 2404, in greedy_search
      outputs = self(
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
      return forward_call(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
      output = module._old_forward(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/unsloth/models/mistral.py", line 211, in MistralForCausalLM_fast_forward
      outputs = self.model(
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
      return forward_call(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
      output = module._old_forward(*args, **kwargs)
    File "/mnt/d/Prace/transformers/prompting-venv-wsl/lib/python3.10/site-packages/unsloth/models/llama.py", line 602, in LlamaModel_fast_forward
      layer_outputs = decoder_layer(
    File "/mnt/d/Prace/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
      return self._call_impl(*args, **kwargs)
    File "/mnt/d/Prace/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
      return forward_call(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
      output = module._old_forward(*args, **kwargs)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/unsloth/models/llama.py", line 398, in LlamaDecoderLayer_fast_forward
      hidden_states = fast_rms_layernorm(self.input_layernorm, hidden_states)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/unsloth/kernels/rms_layernorm.py", line 150, in fast_rms_layernorm
      out = Fast_RMS_Layernorm.apply(X, W, eps)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/torch/autograd/function.py", line 553, in apply
      return super().apply(*args, **kwargs)  # type: ignore[misc]
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/unsloth/kernels/rms_layernorm.py", line 106, in forward
      _rms_layernorm_forward[(n_rows,)](
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/triton/runtime/jit.py", line 532, in run
      self.cache[device][key] = compile(
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/triton/compiler/compiler.py", line 614, in compile
      so_path = make_stub(name, signature, constants, ids, enable_warp_specialization=enable_warp_specialization)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/triton/compiler/make_launcher.py", line 37, in make_stub
      so = _build(name, src_path, tmpdir)
    File "/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/triton/common/build.py", line 106, in _build
      ret = subprocess.check_call(cc_cmd)
    File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmpbd9p8vsg/main.c', '-O3', '-I/mnt/d/transformers/prompting-venv-wsl/lib/python3.10/site-packages/triton/common/../third_party/cuda/include', '-I/usr/include/python3.10', '-I/tmp/tmpbd9p8vsg', '-shared', '-fPIC', '-lcuda', '-o', '/tmp/tmpbd9p8vsg/_rms_layernorm_forward.cpython-310-x86_64-linux-gnu.so', '-L/usr/lib/wsl/lib']' returned non-zero exit status 1.
accelerate==0.27.2
aiohttp==3.9.3
aiosignal==1.3.1
alembic==1.13.1
aniso8601==9.0.1
annotated-types==0.6.0
anyio==4.3.0
async-timeout==4.0.3
attrs==23.2.0
bitsandbytes==0.42.0
blinker==1.7.0
certifi==2024.2.2
charset-normalizer==3.3.2
click==8.1.7
cloudpickle==3.0.0
contourpy==1.2.0
cycler==0.12.1
dataclasses==0.6
dataclasses-json==0.6.4
datasets==2.18.0
dill==0.3.8
distro==1.9.0
docker==7.0.0
docstring-parser==0.15
entrypoints==0.4
exceptiongroup==1.2.0
filelock==3.9.0
Flask==3.0.2
fonttools==4.49.0
frozenlist==1.4.1
fsspec==2024.2.0
gitdb==4.0.11
GitPython==3.1.42
graphene==3.3
graphql-core==3.2.3
graphql-relay==3.2.0
greenlet==3.0.3
gunicorn==21.2.0
h11==0.14.0
httpcore==1.0.4
httpx==0.27.0
huggingface-hub==0.21.3
idna==3.6
importlib-metadata==7.0.1
itsdangerous==2.1.2
Jinja2==3.1.2
joblib==1.3.2
kiwisolver==1.4.5
Mako==1.3.2
Markdown==3.5.2
markdown-it-py==3.0.0
MarkupSafe==2.1.3
marshmallow==3.21.0
matplotlib==3.8.3
mdurl==0.1.2
mlflow==2.11.0
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
mypy-extensions==1.0.0
networkx==3.2.1
numpy==1.26.3
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.19.3
nvidia-nvjitlink-cu12==12.1.105
nvidia-nvtx-cu12==12.1.105
openai==1.13.3
packaging==23.2
pandas==2.2.1
peft==0.9.0
pillow==10.2.0
protobuf==4.25.3
psutil==5.9.8
pyarrow==15.0.0
pyarrow-hotfix==0.6
pydantic==2.6.3
pydantic_core==2.16.3
Pygments==2.17.2
pyparsing==3.1.1
python-dateutil==2.9.0.post0
pytz==2024.1
PyYAML==6.0.1
querystring-parser==1.2.4
regex==2023.12.25
requests==2.31.0
rich==13.7.1
safetensors==0.4.2
scikit-learn==1.4.1.post1
scipy==1.12.0
sentencepiece==0.2.0
shtab==1.7.0
six==1.16.0
smmap==5.0.1
sniffio==1.3.1
SQLAlchemy==2.0.27
sqlparse==0.4.4
sympy==1.12
threadpoolctl==3.3.0
tiktoken==0.6.0
tokenizers==0.15.2
torch==2.2.0
torchaudio==2.2.0+cu121
torchvision==0.17.0+cu121
tqdm==4.66.2
transformers==4.38.2
triton==2.2.0
trl==0.7.11
typing-inspect==0.9.0
typing_extensions==4.8.0
tyro==0.7.3
tzdata==2024.1
unsloth @ git+https://github.com/unslothai/unsloth.git@d862d053ed7deb2275e47d2aec2f7cd587decf0e
urllib3==2.2.1
Werkzeug==3.0.1
xformers==0.0.24
xxhash==3.4.1
yarl==1.9.4
zipp==3.17.0

`

danielhanchen commented 8 months ago

@pauli31 Sorry on the issue! :( Maybe this might help: https://github.com/pytorch/builder/issues/1318 ? It seems like maybe your torch package is partially broken probably, and so Triton can't seem to link properly - it's best to reinstall Torch and Triton :(

pauli31 commented 8 months ago

Thank you, you are right. It is something with the libraries versions. Unfortunately, I was not able to figure it out. ,

danielhanchen commented 7 months ago

@pauli31 Have you tried our notebooks on Colab or Kaggle if those help?

pauli31 commented 7 months ago

Thank you not, yet I'll try it.

1272870698 commented 2 months ago

请问您 这是否解决了

1272870698 commented 2 months ago

Could you please tell me if this has been resolved?

danielhanchen commented 2 months ago

I made the installation probably a bit easier on the Readme - maybe it might help!