search

tsb0601 / MMVP

260 stars 7 forks source link

bug in evaluate scripts? #1

Closed LightDXY closed 6 months ago

LightDXY commented 6 months ago

hi, thanks for your interesting work, but it seems a small bug in the evaluation file. at https://github.com/tsb0601/MMVP/blob/main/scripts/gpt_grader.py#L69, the num_total plus 1 per line, but the num_correct plus 0/1 per two lines, so the max accuracy would be 0.5?

tsb0601 commented 6 months ago

Hi, Thank you for your interest in our work, and thank you for catching that! We updated the gpt-grader script.