gordonwatts / snowmass-chat

Experiments exploring the US Snowmass Process documents using LLM
Apache License 2.0
2 stars 0 forks source link

Some way to record questions for later recovery as we move forward #18

Closed gordonwatts closed 1 year ago

gordonwatts commented 1 year ago

As a developer I want to see how well this model does on a standard set of questions moving forward.

Approach

  1. Create a yaml file that contains the questions
  2. Add a command that will "ask" each question and record the results in a second yaml file.
    • The yaml file will contain the questions, a description of the run, and the answers, and a title of the run.
  3. A command that will turn the questions and answers into a table, with headers the run.