🗺️ Session Tracking

mikeldking commented 8 months ago

As a user of phoenix, I want to be able to associate multiple traces (e.g. conversational flows) under a single session_id. This way I can track the back and forth between a user and a chat completion endpoint (e.g. visualize the back and forth).

In addition to tracking a session.id, we might also want to track session metadata such as user_id etc.

As a developer building a chat or agent application, I want to be able to track user interactions with my application. Notably if my application keeps track of user interactions under a single “session”, I want to be able to view the user interactions at a higher level rather than a trace.

OpenInference Semantic Convention changes

session
- .id the application provided correlation id

OpenInference instrumentation changes

New “openinference-instrumentation” package

contains utilities around correlating spans to sessions

from openinference.instrumentation.session import using_session, set_session, clear_session

# Using a context manager
With using_session(id: “my-unique-id”):
       // Invoke application code here

# implicit
set_session(id: "my-session-id")
// run code here
clear_session(id: "my-session-id")

Proposed solution is to leverage trace context to set “inheritable” attributes that would be added to each span that is nested below a session context. This means that the setting of the session

Open questions:

Threading @axiomofjoy
What if a person starts another session (session collision) @axiomofjoy

Custom Instrumentation

 Custom instrumentation would also have to inherit the attributes in a simple way. We need to provide convenience to pull these attributes and attach them to the span.

from openinference.instrumentation import get_context_attributes

def do_work():
    with tracer.start_as_current_span("span-name", get_context_attributes()) as span:
        # do some work that 'span' will track
        print("doing some work...")
        # When the 'with' block goes out of scope, 'span' is closed for you

Architecture

Under the above, session.id would correlate to a trace (not a span) to avoid the danger of spans that get captured in the absence of a session.id. This must be the assumption when querying for sessions.

Milestone 1 - Instrumentation

[x] #2680
Session Tracking
[x] #2755
[x] #2693
[x] #2678
[x] #2689
[x] #2701
[x] #2697
[x] #3022
[x] #2764
[x] #2759
[x] #2773
[x] #2774
[x] #2775
[x] #2760
[x] #2756
[x] #2765
[x] #2766
[x] #3074
[x] #3076
[x] #3077
[x] #3075
[x] [sessions] update instrumentor to be session-aware
[x] [sessions] update instrumentor to be user-aware
[x] [sessions] update instrumentor to be metadata-aware
[x] [sessions] update instrumentor to be tag-aware
[x] #2761

Instrumentation Documentation

[x] #2767
[x] #2768
[x] #2769
[x] #2770
[x] #2772
[x] #2771
[x] #2758
[x] #2754
[x] #2706

Milestone 2 - Sessions in the UI

Engineering Leads: @RogerHYang @Parker-Stafford

Server Support for Sessions

[x] [sessions] track session ID for a given trace in ingestion
[x] #4854
[x] #5066
[x] #5347
[x] #5355
[x] #5537

Sessions User-Interface

[x] #4884
[x] #4958
[x] #4959
[x] #5008
[x] #5009
[x] #5007
[x] #5278
[x] #5354
[x] #5356
[x] #5357
[x] #5358

Sessions API

[x] #5010
[x] #5011
[x] #5058
[x] #5013
[x] #5014
[x] #5059
[x] #5012
[x] #5279

Milestone 2.5 Sessions JavaScript Support

JavaScript

[x] #3811
[x] #3042
[x] #5084
[x] #3810
[ ] #5085

Milestone 3 - Evaluations / Annotations

As a user I might want to evaluate how a session went.

[ ] [sessions][evals] session evals user-stories
[x] #4899

axiomofjoy commented 8 months ago

With respect to threading and async, OpenTelemetry by default uses contextvars to ensure the context is unique to each thread/ task out of the box. So it's safe for a session context manager to interact with these APIs.

https://github.com/open-telemetry/opentelemetry-python/blob/721beb8b530e7a830c1e27b70c2fb9af6465baf1/opentelemetry-api/src/opentelemetry/context/contextvars_context.py#L19

DixitAdh commented 2 months ago

@mikeldking This looks like a great feature and looking forward to try it out. Let me know if you need any support on testing this. I have a phoenix server deployed in azure containers and logging the traces via databricks notebook. This all is done for RAG chain which is built using langchain.

Arize-ai / phoenix