Open areibman opened 3 days ago
Looking at this, right now it looks like the system handles only sync chat completions so it doesn't seem to be anything hard or particularly taxing to work on!
Having observability handlers for LLMEvent, ActionEvent and ActionEvent seems simple enough, but I did have an idea I wanted to get greenlit! I'm thinking about putting this as a sort of simple wrapper and basically making it so if you set more than one provider on the AI Suite side it automatically initializes multi sessions under (providername.agentops_session) and logs the different sessions within each! Users can set a (start_multisession) bool to false to stop this and have it all go under one session.
I'm halfway through right now and am at the handle_response area; would you rather the entire output is logged with AgentOps as one big message with the different providers all working or the different choices as separate outputs? Or would you rather have an option to toggle between??
💡 Feature Description and Proposed Solution
A new LLM chat framework called AI Suite just launched from Andrew Ng's team https://github.com/andrewyng/aisuite
Adding observability callbacks would be a huge value-add for their project. This would involve:
🤔 Related Problem
No response
🤝 Contribution