Closed githubpradeep closed 11 months ago
Did you install autogptq from https://github.com/ChenMnZ/AutoGPTQ-bugfix?
git clone https://github.com/ChenMnZ/AutoGPTQ-bugfix
pip install -v .
I tried installing the above autogptq but it kept failing installation
The original cuda kernel of autogpt have some bugs, so you should struggle to install it through the bug-fixed version (https://github.com/ChenMnZ/AutoGPTQ-bugfix), then you can obtain correct outputs.
ok i tried it using a smaller model and it worked after installing
@ChenMnZ May I ask what is the bug that you fix? Could you refer to the specific modified file?
i tried running the falcon notebook provided and it didnt generate any coherent sentences it just generated junk characters