search

tloen / llama-int8

Quantized inference code for LLaMA models
GNU General Public License v3.0
1.05k stars 105 forks source link

Further detail needed - installing bitsandbytes from source #16

Open chrisbward opened 1 year ago

chrisbward commented 1 year ago

Not usually familiar with installing python modules outside of pip install -r requirments.txt. just wondering how I would go about the install of this dependency within venv and not conda.

Building the tool shouldn't be an issue, but just wondering how to go about integration - where does it belong?

Cheers!

chrisbward commented 1 year ago

okay, scrap that - I added swap and tried again, just got the 13B working on a 3090 Ti