search

johnsmith0031 / alpaca_lora_4bit

MIT License
534 stars 84 forks source link

Quick start docker image fails to start (cannot import 'Linear8bitLt') #92

Open schwab opened 1 year ago

schwab commented 1 year ago

I'm running into this when attempting to run the docker install. It specifically mentions Triton not found, but I know the Dockerfile includes Triton. Seems like maybe there is a version conflict somewhere. Is there a work-around for this? I'm not familiar with Triton or Linear8bitLt, so not sure what else to check on this.

OS : Ubuntu (Pop Os)

lsb_release -a No LSB modules are available. Distributor ID: Pop Description: Pop!_OS 22.04 LTS Release: 22.04 Codename: jammy

Cuda driver version: 11.7 Nvidia driver version : 515.65.01

------------------------------------------------------------------ Run Results ------------------------------------------------------ docker run --gpus=all -p 7860:7860 --name alpaca_lora_4bit alpaca_lora_4bit

========== == CUDA ==

CUDA Version 11.7.0

Container image Copyright (c) 2016-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

DEPRECATION NOTICE!

THIS IMAGE IS DEPRECATED and is scheduled for DELETION. https://gitlab.com/nvidia/container-images/cuda/blob/master/doc/support-policy.md

Triton not found. Please run "pip install triton". Traceback (most recent call last): File "/alpaca_lora_4bit/text-generation-webui/server.py", line 1, in import custom_monkey_patch # apply monkey patch File "/alpaca_lora_4bit/text-generation-webui/custom_monkey_patch.py", line 6, in from monkeypatch.peft_tuners_lora_monkey_patch import replace_peft_model_with_gptq_lora_model, Linear4bitLt File "/alpaca_lora_4bit/text-generation-webui/monkeypatch/peft_tuners_lora_monkey_patch.py", line 7, in from peft.tuners.lora import is_bnb_available, Linear, Linear8bitLt, LoraLayer ImportError: cannot import name 'Linear8bitLt' from 'peft.tuners.lora' (/root/.local/lib/python3.10/site-packages/peft/tuners/lora.py)

satolone commented 1 year ago

I came across a similar error recently when working with the docker container. Linear8bitLt is imported from the bitsandbytes library. I had to add bitsandbytes library to the requirements.txt file and rebuilt the docker image to fix the issue for me.

johnsmith0031 commented 1 year ago

Thanks for reporting, fixed the bug