search

OpenBB-finance / openbb-agents

R&D playground to play with agents and OpenBB
75 stars 15 forks source link

[Feature] Refactor to treat Agents as an installable package #2

Closed DidierRLopes closed 7 months ago

mnicstruwig commented 7 months ago

Excited to see this! This is exactly what my next step was going to be :) Moving this agent over to it's own repo (we can probably re-use this one), and then to start including evaluation metrics based on the RAG triad (from this short course I saw you tweet out -- it was super valuable / interesting as a starting point!).

The idea is, just like we QA test code, we should be QA testing our agents / RAG pipelines based on a suite of evaluation tasks to see whether performance is improved / diminshed.

Bonus points is to throw an API in front of it, and make it paramaterizable so that it's easy to load functions, then it can be integrated into many different things. Will be working on this soon!

DidierRLopes commented 7 months ago

@mnicstruwig was about to ask, can you take over this one? Either keep working on top of this or close it and start working on a new branch.

I put this together basically so that we could call this in 1 go from Terminal Pro, but ideally, each section is a function and right now some code blocks are just dropped in the openbb_agent function.

I also think that we should test more with all openbb_tools being in the vector store since that would allow us to simply give the prompt. I had good results on this yesterday but sometimes it fails, I think maybe there's something to be done where we improve the docs+output fields into embeddings? Maybe a similar trick to the one you've done for agents with keywords or so