TanGentleman / Augmenta

Automate RAG-powered workflows
MIT License
1 stars 0 forks source link

Construct evaluation chains #18

Open TanGentleman opened 2 months ago

TanGentleman commented 2 months ago

These have been in the works for a while. This is useful for multi-step tasks where validation at one stage is useful. It'll prevent instances of irrelevant context being passed to a chain with the LLM hallucinating or failing at a user-facing step.

This will require a bit of architecture that I've gotten comfy with in the notebooks. It will need few shot examples, as well as a schema validation. In many of these cases, a boolean output is sufficient, but enforcing json will be better so I can handle nuanced cases/errors in the future.