Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development
https://llama2-accessory.readthedocs.io/
Other
2.72k stars 176 forks source link

Eval results do not match the ones in the paper #207

Open chau25102001 opened 1 month ago

chau25102001 commented 1 month ago

Hi, I tried to run the evaluation code for refCOCO, refCOCO+ and refCOCOg with SPHINX-1k. The results were lower than the ones reported in the paper (see attached pic). What could be the reason here? I set the temperature to 0.1, top_p to 0.75 and no quantization was used. Thanks for your great work ! Screenshot from 2024-10-25 09-09-36