span events doesn't show up in the Xray trace segments

[kullu-ashish]

Is your feature request related to a problem? Please describe. span.AddEvents on the span in the code instrumenting the OTELHttp. However, these events don't show up on the Xray traces segments.

Sample code -

func genericHandler(w http.ResponseWriter, r *http.Request, endpoint string) {
ctx := r.Context()
span := trace.SpanFromContext(ctx)

    commonLabels := []attribute.KeyValue{
            attribute.String("labelA", "chocolate"),
            attribute.String("labelB", "raspberry"),
            attribute.String("labelC", "vanilla"),
    }

    span.AddEvent("Test Span A", trace.WithAttributes(commonLabels...))
    time.Sleep(1 * time.Second)
    span.AddEvent("Test Span B")
    span.AddEvent("handling this http request", trace.WithAttributes(attributeEndpoint.String(endpoint)))

    resp, err := httpClient.Get(endpoint)
    if err != nil {
            w.WriteHeader(http.StatusBadRequest)
            w.Write([]byte(fmt.Sprintf("Failed to make request.\n%v", err)))

            msg := fmt.Sprintf("failed to make request.\n%v", err)
            log.Print(msg)

            span.AddEvent("error occurred", trace.WithAttributes(attributeError.String(msg)))
            return
    }
    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)

    if err != nil {
            w.WriteHeader(http.StatusBadRequest)
            w.Write([]byte(fmt.Sprintf("Failed to read body.\n%v", err)))
            msg := fmt.Sprintf("failed to read body.\n%v", err)
            log.Print(msg)

            span.AddEvent("error occurred", trace.WithAttributes(attributeError.String(msg)))
            return
    }

    log.Print(fmt.Sprintf("%v response body: %v", endpoint, string(body)))
    span.AddEvent("body", trace.WithAttributes(attributeReqBody.String(string(body))))
    span.AddEvent("response body", trace.WithAttributes(attributeReqBody.String(string(body))))

    w.Write(append([]byte("Body: "), body...))

}

Only first segment which is the serviceB name shows up but I added 4 events, they don't show up in traces(AWS Xray).

Describe the solution you'd like When you add the span events on the trace, it should create new segments on AWS Xray traces.

Describe alternatives you've considered There is not alternative solution at the moment.

Additional context Added

[willarmiros]

Hi @kullu-ashish,

Unfortunately there is no equivalent to the OpenTelemetry Event in the X-Ray data model so events are dropped when sending OTel spans to the X-ray backend. We will update our docs to reflect this. At this time we are unable to provide an ETA for event support, but will update this issue if there are any updates.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[willarmiros]

Docs have been updated here: https://aws-otel.github.io/docs/getting-started/x-ray#configuring-the-aws-x-ray-exporter

We are working with the X-Ray backend team to update our data model to support span Events as first-class citizens. Will leave this open to track that progress.

[meijeran]

Is there already an update on this?

[atshaw43]

@meijeran I'm afraid I do not have an update to share on the topic. We do not currently support events in X-Ray. I have no ETA on when this functionality will be supported.

[kcrandall]

Is there any update on this? Spent a while trying to figure out why my events werent showing up in AWS managed grafana and i think this is it (using open telemetry aws distro as sidecar with otlp exporter, which is properly being fed events).

Is it being worked on still or has events support been abandoned by xray? Don't expect a solid release date, but this has been open for over 2 years so kinda wanted to know if this is a 2030s feature or something that is actively being worked on by the x-ray team.

[vastin]

The limit support of span event is added in #27232 and documented for HTTP error events. The generic span events support is still in roadmap and there is no ETA yet.

[ishworg]

if this is a 2030s feature or something that is actively being worked on by the x-ray team.

@kcrandall, the thing is that pretty much all serious tracing backends are ahead of xray these days.

Almost everyone I have spoken to uses other tracing backends apart from this toy called xray.