kbressem / medAlpaca

LLM finetuned for medical question answering
GNU General Public License v3.0
474 stars 54 forks source link

Eval usmle #11

Closed kbressem closed 1 year ago

kbressem commented 1 year ago

Rewrite of the inference class, it now prompts the models during inference the same way as they were prompted during training.

Redesigned the evaluation scripts and added some simple "guardrails" to the model, so the question answering output is more standartized.