search

jenkinsci / opentelemetry-plugin

Monitor and observe Jenkins with OpenTelemetry.
https://plugins.jenkins.io/opentelemetry/
Apache License 2.0
100 stars 53 forks source link

Duration as span attribute for Pipeline Step Span #940

Closed rangamani54 closed 2 weeks ago

rangamani54 commented 2 months ago

What feature do you want to see added?

Right now, we have a span attribute for Pipeline duration which is ci.pipeline.run.durationMillis, can we have a similar kind of attribute for pipeline step spans as well? I can see the duration of the span as metadata in tempo, but can we have it as a span attribute?

Upstream changes

No response

Are you interested in contributing this feature?

No response

cyrille-leclerc commented 2 months ago

Can you please share with us why you use the attribute ci.pipeline.run.durationMillis rather than the duration of the build span?

I forgot why we ci.pipeline.run.durationMillis instead of just relying on the duration of the BUILD - *** span. I'm wondering if there is somethign related to the time spent in the build queue.

rangamani54 commented 2 months ago

But the duration is not the span attribute, so that we can't use somewhere else like generating span metrics.

And also in the similar way ci.pipeline.rundurationMillis We need it for child/step spans.

But anyway, I found to do with transform with context span to calculate span duration and set it as an attribute.

mrh666 commented 2 months ago

Hi @cyrille-leclerc , we badly need to have those trace attributes as a metrics in Dynatrace: ci.pipeline.id ci.pipeline.name ci.pipeline.run.* ci.pipeline.type

and we didn't find any way to pass those to Dynatrace. I can't define calculated metrics based on request attributes, because we have only trace attribute. And all existent ci.pipeline.run.* metrics are counters and doesn't have dimensions based on which I can visualize/notify/get metric event for Jenkins build.

I'll highly appreciate if you can tell me if it's doable or not? And point me to some doc or tell me if we can have those in Dynatrace.

mrh666 commented 2 months ago

@cyrille-leclerc I'll rephrase, on that page https://plugins.jenkins.io/opentelemetry/ there is a screenshot of kibana https://raw.githubusercontent.com/jenkinsci/opentelemetry-plugin/master/docs/images/kibana_jenkins_overview_dashboard.png with all graphs I need, e.g. job duration, failed steps, long steps, etc. How can we get those metrics exported to Dynatrace?

rangamani54 commented 2 months ago

I made a Jenkins Build Dashboard from span metrics generated from Tempo. I will soon publish that dashboard with an article

mrh666 commented 2 months ago

@rangamani54 Any chance you can point me to that Tempo doc?

rangamani54 commented 2 months ago

Metrics generator is a component in tempo, you can generate metrics from spans https://grafana.com/docs/tempo/latest/metrics-generator/

mrh666 commented 2 months ago

@cyrille-leclerc any input on the subject?

cyrille-leclerc commented 1 month ago

Please see if we can follow up this conversation on:

cyrille-leclerc commented 2 weeks ago

Please use the ci.pipeline.run.duration{ci.pipeline.id="<<pipeline full name>>", ci.pipeline.result="<<SUCCESS, UNSTABLE, FAILURE, NOT_BUILT, ABORTED>>"} histogram metric we have just released. ℹ Use the otel.instrumentation.jenkins.run.metric.duration.allow_list and otel.instrumentation.jenkins.run.metric.duration.deny_list to specify the pipelines for which you want to capture the run duration, other pipelines will be aggregated in the ci.pipeline.id="#other#" time series.

See documentation https://github.com/jenkinsci/opentelemetry-plugin/blob/main/docs/monitoring-metrics.md#build-duration

I'm marking your enhancement request as solved. Please open new enhancement requests if needed.