search

confident-ai / deepeval

The LLM Evaluation Framework
https://docs.confident-ai.com/
Apache License 2.0
2.54k stars 180 forks source link

Evaluation Dataset Generation #530

Open penguine-ip opened 5 months ago

penguine-ip commented 5 months ago

Evaluation dataset generation is coming to deepeval by end of this week. For this feature, we're looking at the following:

  1. Allow users to generate test cases based on their knowledge base
  2. Allow users to choose how to chunk their knowledge base
  3. Allow users to specify how many test cases to generate
  4. Allow users to complicate test cases to make them more realistic (https://arxiv.org/pdf/2304.12244.pdf, https://mlabonne.github.io/blog/notes/Large%20Language%20Models/phi1.htm)

Feedback, suggestions, and contributions for any of the points above are welcomed 😊

piseabhijeet commented 4 months ago

link not found: https://mlabonne.github.io/blog/notes/Large%20Language%20Models/phi1.htm