search

UKGovernmentBEIS / inspect_ai

Inspect: A framework for large language model evaluations
https://UKGovernmentBEIS.github.io/inspect_ai/
MIT License
400 stars 41 forks source link

expert answer not showing in metadata in view #16

Closed Lovkush-A closed 1 month ago

Lovkush-A commented 1 month ago

I tried the basic hello world example and in the metadata tab for all the rows, it is missing the expert's answer [EXPERT]: {criterion} in between [Question] and [Submission]:

image

The assistant'a output refers to expert answer, which means it must be a bug in how the string is being displayed, rather than bug in evaluation itself.

aisi-inspect commented 1 month ago

Thanks for reporting! We can repro this locally and are investigating (it's likely a vagary of the markdown renderer we are using)

aisi-inspect commented 1 month ago

Resolved with https://github.com/UKGovernmentBEIS/inspect_ai/commit/140c3d63ad42125213bffaa799147135c81857f3