search

EQ-bench / EQ-Bench

A benchmark for emotional intelligence in large language models
MIT License
180 stars 13 forks source link

'BitsAndBytesConfig' object has no attribute 'get_loading_attributes' #19

Closed Abdullah-kwl closed 6 months ago

Abdullah-kwl commented 6 months ago

The parameters which I pass in config.cfg file is: myrun2, ChatML, TheBloke/WestLake-7B-v2-GPTQ, , 4bit, 1, transformers, , ,

I am using the model id "TheBloke/WestLake-7B-v2-GPTQ" It is GPTQ model, and I am facing the error in finding its score on EQ-Bench. please guide me can we calculate the eq-bench score of a GPTQ model such as "TheBloke/WestLake-7B-v2-GPTQ", how can I calculate its eq score? How can I solve this error? i want to find its eq-bench score.

Screenshot 2024-03-11 115915

sam-paech commented 6 months ago

Tbh I haven't tried loading GPTQ models directly with transformers. The supported way to do this with EQ-Bench is using oobabooga. The config for that would be something like:

myrun2, ChatML, TheBloke/WestLake-7B-v2-GPTQ, , None, 1, ooba, --loader AutoGPTQ, --include "*"

Abdullah-kwl commented 6 months ago

I have tried again with this,

The parameters that I pass in config.cfg file is: myrun2, ChatML, TheBloke/WestLake-7B-v2-GPTQ, , None, 1, transformers, , ,

It started working almost 60% of the test was done in 30 minutes, it's taking too much time I did not complete the eq-test, but it was working, please test the eq-bench with the GPTQ model using Transformer, and see why it takes to much time. if there is any issue resolve that.