search

TanGentleman / Augmenta

Automate RAG-powered workflows
MIT License
1 stars 0 forks source link

Construct evaluation chains #18

Open TanGentleman opened 7 months ago

TanGentleman commented 7 months ago

These have been in the works for a while. This is useful for multi-step tasks where validation at one stage is useful. It'll prevent instances of irrelevant context being passed to a chain with the LLM hallucinating or failing at a user-facing step.

This will require a bit of architecture that I've gotten comfy with in the notebooks. It will need few shot examples, as well as a schema validation. In many of these cases, a boolean output is sufficient, but enforcing json will be better so I can handle nuanced cases/errors in the future.

TanGentleman commented 4 months ago

I've gotten much more practice with making eval chains and composing them into more nuanced workflows! I want to experiment with routing a bit more, then we'll start making some proper multi-step chains that call specified functions.