Document properly the LLM output file json format - Githubissues

kasnerz / factgenie

A Toolkit for Annotating and Visualizing LLM Hallucinations

MIT License

1 stars 0 forks source link

Document properly the LLM output file json format #38

Open oplatek opened 3 days ago

oplatek commented 3 days ago

@oplatek used this example as documentation of how the factgenie/outputs/DATASET/SPLIT/MODEL_NAME.json files should be structured.

https://github.com/kasnerz/factgenie/pull/35/files#diff-0fed43a3a8e518bc5e5b80a880afc1206a5bafb7adc34be77f248757d9531f03

the example above loads poorly but the same docs is also at https://practicald2t.github.io/pages/shared_task_evaluation_script.

[ ] move the documentation to README.md/top-level documentation
[ ] describe it more formally, e.g., in JSON schema (and test it?)