UKGovernmentBEIS / inspect_ai

Inspect: A framework for large language model evaluations
https://inspect.ai-safety-institute.org.uk/
MIT License
565 stars 96 forks source link

Typo in @scorer decorator #20

Closed novatc closed 4 months ago

novatc commented 4 months ago

Just a small, minor details I've been stumbling over is the typo in a couple of examples on the website:

@scorer(metrics=[accuracy(), bootstrapstr()]) -> @scorer(metrics=[accuracy(), bootstrapstd()])

found it here for example: Scorers

aisi-inspect commented 4 months ago

Thanks for pointing this out (fixed and should be republished soon).

Apologise for the inconvenience and do let us know if you see others :-)

aisi-inspect commented 4 months ago

Resolved with https://github.com/UKGovernmentBEIS/inspect_ai/commit/140c3d63ad42125213bffaa799147135c81857f3