(Exporter/Geneva) Support ActivityEvents

[cijothomas]

Issue with OpenTelemetry.Exporter.Geneva

• OpenTelemetry.Exporter.Geneva 1.2.6

Is this a feature request or a bug?

• [x] Feature Request
• [ ] Bug

var eventTags = new Dictionary<string, object?>
{
    { "foo", 1 },
    { "bar", "Hello, World!" },
    { "baz", new int[] { 1, 2, 3 } }
};
activity?.AddEvent(new("Gonna try it!", DateTimeOffset.Now, new(eventTags)));

The event above is ignored by the Exporter.

[notheotherben]

Hi, passing along some context on why this matters for our team:

• Span Events (i.e. ActivityEvents) are the standard way that exceptions are handled by OpenTelemetry library implementations across the board, and our engineers are used to being able to use Activity.Current.RecordException(ex) to capture this information quickly and easily.
  • The fact that this can be handled automatically by the OpenTelemetry instrumentation (i.e. without us needing to manually inject try/catch blocks is also a significant maintainability win for us). Currently we're required to add manual try/catch blocks anywhere we'd like to record exception information for a span (which is all of them) just to emit the logging that would otherwise be handled by the span's ActivityEvent anyway.
  • Note that we've tried to work around some of these limitations by adding an Activity processor which emitted logging for span events, however the inability to explicitly provide the log timestamp or the associated parent activity made this significantly less valuable than it would otherwise have been and we ended up abandoning this path.
• Local testing with tools like Jaeger's one-box include support for Span Events out of the box (and don't include any support for things like logging) which means a couple of things:
  • Engineers who write their code using span events get a really great experience locally, and are surprised when they get to production and don't have any of that context available to then.
  • Engineers who use ILogger in their code have a painful time testing things locally, as they need to manually correlate logs against the traces they're looking at (with none of the contextual logging event support available from Span Events in Jaeger). Note that we've hacked around this by adding an ILoggerProvider in our local test environment which appends span events - but it's a hack and something that regularly causes us to run into trouble.
  • When using the console exporters in a testing environment, span events allow us to quickly visualize the context of what occurred during a span's execution without the complexity of trying to associate logs (which are emitted instantly and therefore appear much earlier in the logging) with the span context (which is emitted at the end of the span and much later in the logging) - making debugging test failures substantially harder than it needs to be.
  • Being able to explicitly attach logs to one span or another when multiple are available (we're running a job orchestrator and each job execution gets its own root span, but there are times during the execution that we want to write event context to the job span and the orchestrator span separately, the implicit association with ILogger hampers that.

[giulianob]

@cijothomas Any update on this issue? We also need events to be sent to Geneva.

[NateB2]

@cijothomas - any updates? We're incorporating OpenTelemetry and not being able to export events is a major disappointment for all the reasons @notheotherben mentioned.

[DeanJustus]

Hi folks, any ack on future development or plans for this?

[jaeyoon21]

Dear folks, @cijothomas Our project need ActivityEvents to be shown in geneva. Any update on this issue. Thanks

[emmayspark]

Dear team & @cijothomas, our team is also working on integrating Otel into our systems. Our current logging system uses internal SDK that is deprecating soon, and this supports Exception Telemetry. Because of the limitation of Geneva exporter, we have to find a workaround to log all exceptions manually (but still limited). We need this feature to get exceptions from instrumentation library. Hopefully this feature is considered in the upcoming planning! Thank you.

[cijothomas]

No update on this - i.e this is not coming in the next 6 months.

Consider the following workarounds:

1. If you own the code which produces ActivityEvents, use ILogger instead. Logs are automatically correlated to Activity, so a similar experience to ActivityEvents can be achieved. Be aware that, ILogger logs are not subject to sampling. (but ActivityEvents are part of Activity itself, so sampling affects ActivityEvents as well)
2. If you do not own the code which produce ActivityEvents, use an ActivityProcessor, and convert the ActivityEvents to ILogger logs.

Neither is ideal, but just shared some workarounds.

[cataggar]

I see that Activity.AddException now creates an ActivityEvent. It would be nice if they were sent to Geneva. We do control the code, so we will switch to ILogger.LogError(ex, msg).