UKGovernmentBEIS / inspect_ai

Inspect: A framework for large language model evaluations
https://inspect.ai-safety-institute.org.uk/
MIT License
565 stars 96 forks source link

log viewer: format target using markdown #44

Closed vsiva closed 3 months ago

vsiva commented 3 months ago

In the inspect log viewer, the Scoring tab for a sample shows the Target and Answer in two columns. The Answer is rendered via markdown, but the target isn't. This makes it a little harder to view longer target answers, esp. one that use model-graded-qa and have some code formatting in them.

It looks like this is implemented in SampleScoreView.mjs which includes the two columns like this:

            <td style=${{ paddingRight: "2em", paddingLeft: "0" }}>
              ${arrayToString(sample?.target || "none")}
            </td>

            <td>
              <${MarkdownDiv} 
                class="no-last-para-padding"
                markdown=${
                  shortenCompletion(answerForSample(sample)
              )} style=${{ paddingLeft: "0" }}/>
            </td>
aisi-inspect commented 3 months ago

@dragonstyle any reason not to also do marking rendering for target?

dragonstyle commented 3 months ago

There is definitely no reason not to render the target there. I think it also worth considering rendering them in the sample list and header, which I'll tinker with. Should see some changes in an upcoming release!

aisi-inspect commented 3 months ago

This should be resolved with our 0.3.15 release (available on PyPI now).