search

chen700564 / RGB

Other
276 stars 25 forks source link

Need openAI key for evaluating LLM? #1

Closed valdesguefa closed 1 year ago

valdesguefa commented 1 year ago

I would like to thank you for the work you have done and encourage you to continue. I would like to know if an openAI key is required for the evaluation and if it is possible to evaluate models that have undergone quantization.

chen700564 commented 1 year ago

(1) You can evalue the models in evalue.py, which is based on exact matching and free from openai models. To achieve better rejection evaluation and fact evaluation, openai models are needed in fact_evalue.py and reject_evalue.py. (2) You can change models/models.py to use any models or quantized models.