NotImplementedError: Unsloth: unsloth/gemma-2-9b-bnb-4bit not supported yet!

[xyluo25]

model, tokenizer = FastLanguageModel.from_pretrained( model_name = "unsloth/gemma-2-9b-bnb-4bit", max_seq_length = max_seq_length, dtype = None, load_in_4bit = True)

I noticed models are exists on huggingface)

[coenvdgrinten]

Re-installing unsloth in conda worked for me!

pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"

[djannot]

It didn't work for me

[danielhanchen]

@djannot @xyluo25 @coenvdgrinten Try updating Unsloth via:

pip uninstall unsloth -y
pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git

We also have a Colab notebook for Gemma 2: https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing

[djannot]

@danielhanchen I've tried this already, but it doesn't work unfortunately.

Collecting git+https://github.com/unslothai/unsloth.git
  Cloning https://github.com/unslothai/unsloth.git to /tmp/pip-req-build-q84r99wn
  Running command git clone --filter=blob:none --quiet https://github.com/unslothai/unsloth.git /tmp/pip-req-build-q84r99wn
  Resolved https://github.com/unslothai/unsloth.git to commit 5ab565fb2c811d0b85d68dadd2ac1b32dee05e8b
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: unsloth
  Building wheel for unsloth (pyproject.toml) ... done
  Created wheel for unsloth: filename=unsloth-2024.7-py3-none-any.whl size=124218 sha256=fd1ea03a8e61ce7a8b12ed0851b27c19a640743f43f9f5f1c7e08e74d79b6205
  Stored in directory: /tmp/pip-ephem-wheel-cache-d4iaw7se/wheels/ed/d4/e9/76fb290ee3df0a5fc21ce5c2c788e29e9607a2353d8342fd0d
Successfully built unsloth
Installing collected packages: unsloth
Successfully installed unsloth-2024.7

[djannot]

@danielhanchen I found the problem. I forgot my program was in a directory where I cloned the unsloth repo, so it was using it instead of the package installed with pip.