search

ai-cfia / ailab-db

database related scripts and setup
MIT License
1 stars 0 forks source link

Evaluate a Question Quality #74

Open JolanThomassin opened 8 months ago

JolanThomassin commented 8 months ago

We tried using LLM to check how good our questions were, wanting to see if they were suitable for use or not.

The aim here is to see if we can find a method to check how good a question or a sentence is.

_Issue posted by @rngadam in https://github.com/ai-cfia/ailab-db/pull/47#discussion_r1508380724_

JolanThomassin commented 8 months ago

The QnA appears to be good, but occasionally, the questions are off-topic. The QnA generation seems to be functioning correctly; however, the issue lies with the initial document we receive. I'll attempt to obtain a full page instead of a smaller chunk to check if we can get better results.

rngadam commented 7 months ago

The QnA appears to be good, but occasionally, the questions are off-topic. The QnA generation seems to be functioning correctly; however, the issue lies with the initial document we receive. I'll attempt to obtain a full page instead of a smaller chunk to check if we can get better results.

the approach should instead use score to filter scores