GoogleCloudPlatform / genai-databases-retrieval-app

Apache License 2.0
212 stars 72 forks source link

chore: add golden dataset for eval #411

Closed Yuan325 closed 4 months ago

Yuan325 commented 5 months ago

Adding golden datasets that will be used in llm system evaluation.

The golden dataset is separated into multiple types of queries:

jackwotherspoon commented 4 months ago

Just putting this here as a friendly suggestion, can take it or leave it 😄

Long-term open source maintainability can improve drastically by PRs having at a minimum a one or two sentence description (especially for feat PRs but even for all types). Just going over what you are adding and why.

The benefits:

I recently have found this extremely beneficial in Cloud SQL repos (example)

Github also recommends this themselves

kurtisvg commented 4 months ago

+1 to Jack's comment. We could do a better job on PR descriptions.