search

nicknochnack / Llama2RAG

A working example of RAG using LLama 2 70b and Llama Index
352 stars 110 forks source link

load_in_8bit=True` requires Accelerate: #1

Open andsty opened 1 year ago

andsty commented 1 year ago

ImportError: Using load_in_8bit=True requires Accelerate: pip install accelerate and the latest version of bitsandbytes pip install -i https://test.pypi.org/simple/ bitsandbytes or pip install bitsandbytes` Any help is apprecoated

andsty commented 1 year ago

I have everything installed

vvhanxing commented 12 months ago

I have everything installed I meet this error too,even though i install every thing include accelerate and bitsandbytes

yruslan commented 10 months ago

Downgrading transformers (from this SO answer https://stackoverflow.com/a/76976563/1038282) helped solve the issue:

pip install transformers==4.30