[ENHANCEMENT] Full trace from spanid, incl evaluations, from REST api/python client

[stdweird]

Is your feature request related to a problem? Please describe. We want to provided detailed feedback to "expert" users, and thus want to give them access to the full trace of a eg a query/response flow from a chatbot; but not to phoenix itself. We currently only gather a single spanid from the whole flow, and want to find the trace this spanid belongs too, and retrieve all other spans and some hierarchy info. This ideally also contains the evaluations.

Describe the solution you'd like A rest api where we can post the spanid, and get back as answer all data a trace detail/overlay would be able to show. Hierarchical data can be in nested "dict" structure, or with references between parent/child.

Also an extra method to the python client to retrieve this data.

Describe alternatives you've considered Eg to retrieve the evaluations belong to a span, we have no other way bit to retrieve all evaluations, and filter out everything except the one we need.

[axiomofjoy]

Hey @stdweird, we'll be building out auth in Phoenix. Maybe what you're asking for is the ability to share traces and spans? Or is it important for you to be able to pull this information programmatically?

[stdweird]

@axiomofjoy hmm, ideally we have both ;)

programmatic access to the evaluations associated with a single spanid is something we will actually need. pulling all evaluation to get single set associated with known spanid, not sure how long this will last.

on the authenticated traces, what are you planning to provide: group based access mapped to projects? some user metatadata in the span, and authenticated user can see all its spans, and nothing else? our use case with the full traces view is that (certain classes of) users can see the traces, but we want to give them a url of some sort to click on and point it to the trace of the answer. now we have the spanid per generated answer (in a chatbot scenario), so we would need a url like "/traces/?spanid=abcdef" to resolve to the trace. we are not going to let users search for the trace.

[axiomofjoy]

on the authenticated traces, what are you planning to provide: group based access mapped to projects? some user metatadata in the span, and authenticated user can see all its spans, and nothing else? our use case with the full traces view is that (certain classes of) users can see the traces, but we want to give them a url of some sort to click on and point it to the trace of the answer. now we have the spanid per generated answer (in a chatbot scenario), so we would need a url like "/traces/?spanid=abcdef" to resolve to the trace. we are not going to let users search for the trace.

We haven't scoped authentication yet so I am not sure, but thanks for the feedback!

To unblock for now, you can query the GraphQL API directly.

from httpx import Client

query = """
  query ($spanId: GlobalID!) {
    span: node(id: $spanId) {
      ... on Span {
        id
        spanEvaluations {
          name
          score
          label
          explanation
        }
      }
    }
  }
"""
span_id = "U3BhbjoxMA=="  # replace with your span id
test_client = Client(base_url="http://localhost:6006")
response = test_client.post(
    "/graphql",
    json={
        "query": query,
        "variables": {
            "spanId": span_id,
        },
    },
)
assert response.status_code == 200
response_json = response.json()
assert response_json.get("errors") is None
print(response_json["data"])

[stdweird]

@axiomofjoy sorry for getting back so late to this. i get an error

graphql query='\n  query ($spanId: GlobalID!) {\n    span: node(id: $spanId) {\n      ... on Span {\n        id\n      \tspanEvaluations {\n          name\n          score\n          label\n          explanation\n        }\n      }\n    }\n  }\n' variables={'spanId': '2437f393a545acfc'} returned errors=[{'message': "Variable '$spanId' got invalid value '2437f393a545acfc'; Expected type 'GlobalID'. 'utf-8' codec can't decode byte 0xfb in position 2: invalid start byte", 'locations': [{'line': 2, 'column': 10}]}]: response=<Response [200]>

any idea what this could be about? also, the spanid you give, what format is this? i am used to either int or the hex version of the int (what is eg used when adding an evaluation via the python client)

[stdweird]

ok, after playing around with GlobalID (and the spanId a str of the int of the span id), i get another error

ERROR:root:graphql query='\n  query ($spanId: GlobalID!) {\n    span: node(id: $spanId) {\n      ... on Span {\n        id\n      \tspanEvaluations {\n          name\n          score\n          label\n          explanation\n        }\n      }\n    }\n  }\n' variables={'spanId': 'U3BhbjoyNjA5ODIyMzI0NTQzMDQwNzY0'} returned errors=[{'message': "(sqlalchemy.dialects.postgresql.asyncpg.Error) <class 'asyncpg.exceptions.DataError'>: invalid input for query argument $1: 2609822324543040764 (value out of int32 range)\n[SQL: SELECT traces.id, traces.project_rowid, traces.trace_id, traces.start_time, traces.end_time, spans.id AS id_1, spans.trace_rowid, spans.span_id, spans.parent_id, spans.name, spans.span_kind, spans.start_time AS start_time_1, spans.end_time AS end_time_1, spans.attributes, spans.events, spans.status_code, spans.status_message, spans.cumulative_error_count, spans.cumulative_llm_token_count_prompt, spans.cumulative_llm_token_count_completion \nFROM spans JOIN traces ON traces.id = spans.trace_rowid \nWHERE spans.id = $1::INTEGER]\n[parameters: (2609822324543040764,)]\n(Background on this error at: https://sqlalche.me/e/20/dbapi)", 'locations': [{'line': 3, 'column': 5}], 'path': ['span']}]: response=<Response [200]>

[stdweird]

so the int value of the spanid i use is way too big. this is the spanid i get from eg a ReadableSpan context.span_id (if i convert this to a hex string and use it to create an evaluation, this works)

[axiomofjoy]

@stdweird I may have lead you astray, it looks like the span ID you need to use this script is not available in the UI.

[axiomofjoy]

We'll likely need to build out a REST route to satisfy this need.