search

UKGovernmentBEIS / inspect_ai

Inspect: A framework for large language model evaluations
https://UKGovernmentBEIS.github.io/inspect_ai/
MIT License
385 stars 41 forks source link

minor typos in dataset docs #21

Closed Lovkush-A closed 1 month ago

Lovkush-A commented 1 month ago

  1. "A sample has an input, a target, an optional id, and ...". This makes it sound like the target is not optional. Two suggestions:

    • change "target" to "optional target"
    • maybe drop the sentence, because the table already explains what a sample is. The sentence also does not mention choices which is in the table.

  2. "Note that samples from datasets without and id field will". "and" should be "an"

  3. "If your samples include choices, then the label should be a numeric index into the available choices rather a letter"

    • 'label' should be 'target'? I'm not sure.
    • 'rather a letter' should be 'rather than a letter'

  4. "you can instead pass a function which takes an index and record". Looks like you only need to pass a record, not an index too?

  5. "See the article on Configuring the AWS CLI for additional details". Missing full-stop at end.

aisi-inspect commented 1 month ago

Thank you for these edits! We'll publish a new version of the docs w/ these corrections in the next few days (and will close this issue once we have done so).

aisi-inspect commented 1 month ago

Resolved with https://github.com/UKGovernmentBEIS/inspect_ai/commit/140c3d63ad42125213bffaa799147135c81857f3