kbressem / medAlpaca

LLM finetuned for medical question answering
GNU General Public License v3.0
491 stars 58 forks source link

Eval usmle #11

Closed kbressem closed 1 year ago

kbressem commented 1 year ago

Rewrite of the inference class, it now prompts the models during inference the same way as they were prompted during training.

Redesigned the evaluation scripts and added some simple "guardrails" to the model, so the question answering output is more standartized.