open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.09k stars 2.38k forks source link

Issue in calls_total metric of spanmetric connector #34126

Open namkattor123 opened 4 months ago

namkattor123 commented 4 months ago

Component(s)

connector/spanmetrics

Describe the issue you're reporting

I installed opentelemetry java agent and using [opentelemetry-collector-contrib] to expose metrics which displayed SPM on JaegerUI. But metric calls_total doesn't increase, it increase and decreas over time.Screenshot 2024-07-17 at 08 47 38

github-actions[bot] commented 4 months ago

Pinging code owners:

namkattor123 commented 4 months ago

My metric graph Screenshot 2024-07-17 at 08 49 53

namkattor123 commented 4 months ago

My SPM monitor on Jaeger Screenshot 2024-07-17 at 08 51 33

crobert-1 commented 4 months ago

Hello @namkattor123, from what I can tell, the metric is the current batch's total, rather than the total count of all spans seen ever. I believe this is on purpose.

I'll defer to code owners though in case there's something I'm missing.

namkattor123 commented 4 months ago

Thank @crobert-1 , This photo is my value of call_totals metric. As documents of spanmetric connector, it's counter type. I don't understand why it can decrease

namkattor123 commented 4 months ago

Screenshot 2024-07-18 at 10 25 47 This is my pipeline config

namkattor123 commented 4 months ago

Screenshot 2024-07-18 at 10 25 28 This is my data value of metric

Frapschen commented 3 months ago

@namkattor123 Please check your spanmetrics connector aggregation_temporality config, if the value is AGGREGATION_TEMPORALITY_DELTA, the call_totals will be gauge metrics.

Frapschen commented 3 months ago

more like this: https://github.com/open-telemetry/opentelemetry.io/issues/4368

namkattor123 commented 3 months ago

I config spanmetric with default, it means that aggregation_temporality config must be AGGREGATION_TEMPORALITY_CUMULATIVE, why it's setten to AGGREGATION_TEMPORALITY_DELTA . Can you explain for me @Frapschen

Frapschen commented 3 months ago

@namkattor123 have you add resource_metrics_key_attributes to your spanmetrics connector? please see https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/21101#issuecomment-1910295920

Frapschen commented 3 months ago

@namkattor123 have you add resource_metrics_key_attributes to your spanmetrics connector? please see #21101 (comment)

@namkattor123 if the above solution does not work for you, you can use the UUID() function in the transform processor to add a unique UUID to metrics attributes in each collector. I have tested this method, it works for me.

github-actions[bot] commented 1 month ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.