open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.02k stars 2.33k forks source link

metrics scopeName error #21082

Closed daoweili closed 1 year ago

daoweili commented 1 year ago

Component(s)

exporter/clickhouse

What happened?

Describe the bug When I use a custom meter and set instrumentationScopeName,but the scopeName is not effective in the clickhouse. i used java agent version is 1.24.0, collector version is v0.75.0

Steps to reproduce image image image

image

Collector version

v0.75.0

Environment information

Environment

OS: (e.g., "Ubuntu 16.04")

OpenTelemetry Collector configuration

extensions:
  health_check:
  pprof:
    endpoint: 0.0.0.0:1777
  zpages:
      endpoint: 0.0.0.0:55679
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  jaeger:
    protocols:
      grpc:
        endpoint: 0.0.0.0:14251
      thrift_compact:  
        endpoint: 0.0.0.0:6931
processors:
  batch:
exporters:
  jaeger:
    endpoint: xxx
    tls:
      insecure: true
  kafka/metrics: 
    brokers:
      - xxx
    protocol_version: 2.0.0
    topic: otel_metrics_test
    encoding: otlp_json
    producer: 
      compression: snappy
  kafka/logs: 
    brokers:
      - xxx
    protocol_version: 2.0.0
    topic: otel_logs_test
    encoding: otlp_json
    producer: 
      compression: snappy
  clickhouse: 
    endpoint: xxx
    ttl_days: 30
    timeout: 5s
    database: xxx
    connection_params: {"dial_timeout":"200ms","max_execution_time":60}
    username: xxx
    password: xxxx
    logs_table_name: otel_logs
    traces_table_name: otel_traces
    metrics_table_name: otel_metrics
    sending_queue: 
      queue_size: 5000
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
service:
  pipelines:
    traces:
      receivers: [otlp, jaeger]
      processors: [resource, batch]
      exporters: [jaeger, clickhouse]
    metrics:
      receivers: [otlp]
      processors: [resource, batch]
      exporters: [kafka/metrics, clickhouse]
    logs:
      receivers: [otlp]
      processors: [resource, batch]
      exporters: [kafka/logs, clickhouse]
  extensions: [health_check, pprof, zpages]

Log output

No response

Additional context

No response

github-actions[bot] commented 1 year ago

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Frapschen commented 1 year ago

Can you use logging exporter to check the metric data that collector actual collect:

exporters:
  logging/detailed:
    verbosity: detailed
daoweili commented 1 year ago

yes, let me try.

partial logs:

ScopeMetrics #5 ScopeMetrics SchemaURL: InstrumentationScope log-service-api 0.0.1 Metric #0 Descriptor: -> Name: index.check.histogram -> Description: -> Unit: -> DataType: Histogram -> AggregationTemporality: Delta HistogramDataPoints #0 StartTimestamp: 2023-04-26 02:22:20.689450221 +0000 UTC Timestamp: 2023-04-26 02:23:20.68944636 +0000 UTC Count: 2 Sum: 13.000000 Min: 5.000000 Max: 8.000000 ExplicitBounds #0: 0.000000 ExplicitBounds #1: 5.000000 ExplicitBounds #2: 10.000000 ExplicitBounds #3: 25.000000 ExplicitBounds #4: 50.000000 ExplicitBounds #5: 75.000000 ExplicitBounds #6: 100.000000 ExplicitBounds #7: 250.000000 ExplicitBounds #8: 500.000000 ExplicitBounds #9: 750.000000 ExplicitBounds #10: 1000.000000 ExplicitBounds #11: 2500.000000 ExplicitBounds #12: 5000.000000 ExplicitBounds #13: 7500.000000 ExplicitBounds #14: 10000.000000 Buckets #0, Count: 0 Buckets #1, Count: 1 Buckets #2, Count: 1 Buckets #3, Count: 0 Buckets #4, Count: 0 Buckets #5, Count: 0 Buckets #6, Count: 0 Buckets #7, Count: 0 Buckets #8, Count: 0 Buckets #9, Count: 0 Buckets #10, Count: 0 Buckets #11, Count: 0 Buckets #12, Count: 0 Buckets #13, Count: 0 Buckets #14, Count: 0 Buckets #15, Count: 0 Metric #1 Descriptor: -> Name: index.check.gauge -> Description: -> Unit: -> DataType: Gauge NumberDataPoints #0 StartTimestamp: 2023-04-26 02:22:20.689450221 +0000 UTC Timestamp: 2023-04-26 02:23:20.68944636 +0000 UTC Value: 100 Metric #2 Descriptor: -> Name: index.check -> Description: check counts -> Unit: 1 -> DataType: Sum -> IsMonotonic: true -> AggregationTemporality: Delta NumberDataPoints #0 Data point attributes: -> test: Str(test) StartTimestamp: 2023-04-26 02:22:20.689450221 +0000 UTC Timestamp: 2023-04-26 02:23:20.68944636 +0000 UTC Value: 2 Metric #3 Descriptor: -> Name: index.up_down_counter -> Description: -> Unit: -> DataType: Sum -> IsMonotonic: false -> AggregationTemporality: Cumulative NumberDataPoints #0 StartTimestamp: 2023-04-23 08:55:20.685227279 +0000 UTC Timestamp: 2023-04-26 02:23:20.68944636 +0000 UTC Value: 1248786

image image

from different libraries, the InstrumentationScope names are the same when inserting into ClickHouse.