search

bigcode-project / starcoder2

Home of StarCoder2!
Apache License 2.0
1.71k stars 158 forks source link

Can't load StarCoder2-3B #6

Closed gsakkas closed 6 months ago

gsakkas commented 6 months ago

Hi,

I am using Python 3.10.8 and I've updated all packages, but I still get this error message, when try to use the model:

ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`

Any ideas what the problem might be?

udaygiri commented 6 months ago

Can you share screenshot of Error?

gsakkas commented 6 months ago

image

That's the whole thing.

udaygiri commented 6 months ago

6

ImportError with bitsandbytes This error indicates that your environment lacks the necessary accelerate and updated bitsandbytes packages required for 8-bit quantization in the Transformers library.

To fix this, you need to install or update these packages. Run the following commands in your terminal.

pip install -U accelerate pip install -U -i https://pypi.org/simple/ bitsandbytes

This will install the latest versions of accelerate and bitsandbytes, ensuring compatibility with the 8-bit quantization feature used in your script.

This may solve problems.

gsakkas commented 6 months ago

Thank you! I have them installed globally but still get the same error. It works when I made new .venv and installed all requirements again.