AgentOps-AI / agentops

Python SDK for agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen
https://agentops.ai
MIT License
1.18k stars 93 forks source link

Session management #252

Open sarath59 opened 3 weeks ago

sarath59 commented 3 weeks ago

In this pull request, we have enhanced the session management and event recording functionalities within the AgentOps framework. These improvements aim to provide more robust and flexible session handling, including session expiration and better management of session-related events. Below is a detailed breakdown of the changes made:

Session Expiration:

Added an expiration_time attribute to the Session class to track session expiration. Implemented the is_expired property in the Session class to check if a session has expired based on the current time. Session Video Recording:

Added the set_session_video method to the Session class to set a URL for the video recording of the session. Enhanced Session Handling in Client:

Updated the start_session method to initialize the session expiration time. Updated the end_session method to handle ending a session with an optional video URL. Improved _handle_unclean_exits to ensure sessions are properly ended even if the process exits unexpectedly. Improved LLM Tracking:

Enhanced LlmTracker to track function calls and manage events more effectively. Improved the handling of streaming and async responses in _handle_response_v0_openai, _handle_response_v1_openai, and _handle_response_cohere methods.

Test Script python Copy code import unittest from uuid import uuid4 from agentops.client import Client from agentops.session import Session from agentops.helpers import get_ISO_time from time import sleep

class TestAgentOpsEnhancements(unittest.TestCase):

def setUp(self):
    self.client = Client(api_key="test_api_key")
    self.session_id = uuid4()

def test_session_creation_and_expiration(self):
    session = Session(session_id=self.session_id, tags=["test_tag"])
    self.assertFalse(session.has_ended)
    self.assertFalse(session.is_expired)

    # Simulate session expiration
    session.expiration_time = get_ISO_time() - 1
    self.assertTrue(session.is_expired)

def test_set_session_video(self):
    session = Session(session_id=self.session_id, tags=["test_tag"])
    session.set_session_video("http://example.com/video.mp4")
    self.assertEqual(session.video, "http://example.com/video.mp4")

def test_start_and_end_session(self):
    self.client.start_session(tags=["test_start_end"])
    self.assertIsNotNone(self.client.current_session_id)

    self.client.end_session("Success", "Test end session", "http://example.com/video.mp4")
    self.assertIsNone(self.client.current_session_id)

def test_unclean_exit_handling(self):
    session = Session(session_id=self.session_id, tags=["unclean_exit_test"])
    self.client._session = session
    self.client._worker = unittest.mock.Mock()

    # Simulate unclean exit
    self.client._handle_unclean_exits()
    atexit._run_exitfuncs()
    self.client._worker.end_session.assert_called_once()

if name == "main": unittest.main() Explanation of the Test Script test_session_creation_and_expiration:

Creates a session and checks that it has not ended and is not expired initially. Simulates session expiration and checks that the session is marked as expired. test_set_session_video:

Creates a session, sets the session video URL, and verifies that the video URL is correctly set. test_start_and_end_session:

Starts a session using the client and verifies that a session ID is set. Ends the session and verifies that the session ID is cleared. test_unclean_exit_handling:

Simulates an unclean exit scenario and verifies that the session end method is called to handle the unclean exit.

siyangqiu commented 3 weeks ago

Hi @sarath59! Thanks for the contribution, but what problem does this change aim to solve? Enhanced LlmTracker to track function calls and manage events more effectively -> You only removed comments? What is an expired session? Our backend doesn't have metadata field, so it wouldn't understand it.

Was this AI generated?

sarath59 commented 3 weeks ago

Hi @sarath59! Thanks for the contribution, but what problem does this change aim to solve? Enhanced LlmTracker to track function calls and manage events more effectively -> You only removed comments? What is an expired session? Our backend doesn't have metadata field, so it wouldn't understand it.

Was this AI generated?

Hey @siyangqiu , Thanks for reviewing my contribution. My motive was to make minor changes to session management and not to make huge changes. I just want to start with this for contributing.

Coming to the expired session the goal was to add functionality to detect and handle these expired sessions to ensure they are managed appropriately and do not cause issues in the system.

I may have pushed the wrong file but my intention was to make above changes to session management. I can push changes again with correct files.

My idea was to start small with contributions and see how it goes. I thought there could be better session management as the whole functionality resides between the sessions.

Coming to the AI generated question. I have drafted the above description based on my thoughts to make it professional but nothing from the code.

Also I would love if you could review the notifications feature i proposed and let me know if you would be interested in me doing that.

thanks

bboynton97 commented 2 weeks ago

Implemented the is_expired property in the Session class to check if a session has expired based on the current time.

i may be blind, but i dont see where this code is in this PR

bboynton97 commented 2 weeks ago

thank you for your contribution!! i think there may be some more work here needed for this PR to be ready to merge. we also need to discuss a bit more on the metadata feature and decide if that's something we want to support.

I've left some comments on the code if you want to respond to those :)