search

paolorechia / learn-langchain

MIT License
275 stars 41 forks source link

Linux install, No module named 'gptq_for_llama' #22

Closed unoriginalscreenname closed 1 year ago

unoriginalscreenname commented 1 year ago

I finally gave up on windows and installed everything on a fresh ubuntu wsl. I have everything setup properly including the nvidia toolkit, however if you try and run the server with 4 bit true, you get this error:

from gptq_for_llama.llama_inference import load_quant ModuleNotFoundError: No module named 'gptq_for_llama'

paolorechia commented 1 year ago

Did you install this package already?

gptq-for-llama @ git+https://github.com/paolorechia/GPTQ-for-LLaMa@cadbacf0dcc18f7c56db54561ad53ba0f8db878c

paolorechia commented 1 year ago

It’s in the requirements.txt

unoriginalscreenname commented 1 year ago

yeah, i've installed all the requirements. I don't know, I'm now spending more time trying to get install and configure things than actually getting to play with any of this stuff. Oobabooga doesn't work on linux either, it's giving me the same error about GPTQ. This whole thing is such a mess. This was a fresh linux install, it just doesn't make sense. I don't understand how people get all this stuff to work so easily.

paolorechia commented 1 year ago

Sorry to hear about the bad experience - I’m using Ubuntu personally , hadn’t had that many problems

It looks like you might need to debug your environment - maybe also delete everything and try again

Another thing you can try is cloning the GPTQ-For-Llama repo and installing it locally

You can enter the directory and do pip install -e .

TestUser @.***> schrieb am So. 7. Mai 2023 um 12:41:

yeah, i've installed all the requirements. I don't know, I'm now spending more time trying to get install and configure things than actually getting to play with any of this stuff. Oobabooga doesn't work on linux either, it's giving me the same error about GPTQ. This whole thing is such a mess. This was a fresh linux install, it just doesn't make sense. I don't understand how people get all this stuff to work so easily.

— Reply to this email directly, view it on GitHub https://github.com/paolorechia/learn-langchain/issues/22#issuecomment-1537382929, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJDFZ6TLO2XHO2ZOAZ7UX3XE5355ANCNFSM6AAAAAAXYYU7AA . You are receiving this because you commented.Message ID: @.***>

unoriginalscreenname commented 1 year ago

it's alright. Yeah, i just did a clean install of the repo and installed using the included script and it still comes back with no gptq_for_llama module when trying to load the quant_loader.

unoriginalscreenname commented 1 year ago

import sys from pathlib import Path sys.path.insert(0, str(Path("GPTQ-for-LLaMa")))

from llama_inference import load_quant

Adding this to the quant_loader seems to help. I also cloned the repo directly into the root folder of the project. I'm still getting an error, but it can at least detect the module now

unoriginalscreenname commented 1 year ago

Yes. this is now working! for whatever reason it's not recognizing the module from the requirements.

paolorechia commented 1 year ago

Nice to hear, I’ll keep this issue open since more people might run into this issue

SunnyBingoMe commented 1 year ago

Hi, I met the same problem and fixed by using @unoriginalscreenname method. But I got another weird error, the running progress goes to another conda venv instead of sticking with the current venv (learn-langchain). Not sure why but I did cloned the previous venv to get learn-langchain. Thus I removed this cloned venv and installed conda venv from the beginning. This issue is gone. Hope this fix someone's issues. BR/sunny