do i need to change the dockerfile to install triton?

[wesleysanjose]

I am trying to run on my ubuntu 20.04 with 2060super

Triton not found. Please run "pip install triton".

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

/root/.local/lib/python3.10/site-packages/torch/cuda/init.py:107: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.) return torch._C._cuda_getDeviceCount() > 0 bin /root/.local/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so /root/.local/lib/python3.10/site-packages/bitsandbytes/cextension.py:33: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable. warn("The installed version of bitsandbytes was compiled without GPU support. " CUDA SETUP: Loading binary /root/.local/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so... Traceback (most recent call last): File "/alpaca_lora_4bit/text-generation-webui/server.py", line 1, in import custom_monkey_patch # apply monkey patch File "/alpaca_lora_4bit/text-generation-webui/custom_monkey_patch.py", line 6, in from monkeypatch.peft_tuners_lora_monkey_patch import replace_peft_model_with_gptq_lora_model, Linear4bitLt ModuleNotFoundError: No module named 'monkeypatch'

[johnsmith0031]

If you're using cuda backend, you do not need to install triton. I think you can add something like this:

export PYTHONPATH=$PYTHONPATH:path of the repo
run webui

[wesleysanjose]

fixed the Dockerfile and reinstalled 525 nvidia driver, i got it run finally