Collect expert 'question - expected answer' pairs for component development and testing

soilwise-he / natural-language-querying

Application component that provides Natural Language Querying (NLQ) services, making knowledge stored in a graph database accessible for e.g. a ChatBot UI.

MIT License

0 stars 0 forks source link

Collect expert 'question - expected answer' pairs for component development and testing #6

Open robknapen opened 5 months ago

robknapen commented 5 months ago

To develop, fine-tune, test and validate the NLQ component a set of "prompt scenarios", or defined pairs of questions with expected answers (including reference to sources that need to have been used/mentioned) is needed. Such a datasets also helps to clarify what is expected from the component, including e.g. context to be used and style of writing in the generated responses.

roblokers commented 3 months ago

We have suggested Fenny/Thaisa to start collecting such scenarios in the upcoming months. Will monitor progress for that.