Error with 4bit-128g model loading

[Magnaderra]

Hi to all!

When I load this model (from directory): https://huggingface.co/OccamRazor/pygmalion-6b-gptq-4bit, I get error:

FileNotFoundError: [Errno 2] No such file or directory: 'C:\\KoboldAI\\models\\pygmalion-6b-gptq-4bit\\quantize_config.json'

Here full bug report: Bug_Report.txt

I previously used KoboldAI's fork by 0cc4m and there was no such error.

Here the debug dump: kobold_debug.zip

[henk717]

Did you select occam gptq as the GPTQ backend? Because for autogptq you need an extra file.

[Magnaderra]

Did you select occam gptq as the GPTQ backend? Because for autogptq you need an extra file.

Yes, I have occam gptq selected

[henk717]

Traced this back to the newer transformers version we are using compared to occam's old branch. We will have to find a way to make the changes required to handle this old model, for the time being you have two options. You can either downgrade to the old transformers by opening the KoboldAI command prompt and typing this : pip install transformers == 4.28 This method will harm your model compatibility but restores this old model, you can also use a different version of the model such as the one in the chat model menu.

[Magnaderra]

pip install transformers == 4.28

Okay, so far that's worked.