search

kbressem / medAlpaca

LLM finetuned for medical question answering
GNU General Public License v3.0
491 stars 57 forks source link

Fix the logic of skipping in eval_usmle #36

Closed hyesunyun closed 1 year ago

hyesunyun commented 1 year ago

The current code would skip the first instance of each of the USMLE step questions. I have fixed the code to consider all the questions and not skip the first question.

The fix could also be the following: if skip_if_exists and question["no"] <= len(answers):

kbressem commented 1 year ago

Thanks for pointing this out!