Collector-exported data mapping data point attributes to unknownFields

[hxliu33]

I am currently using a Collector with an otlphttp exporter to send metrics to a custom backend I am writing in Java, but when I process the request body using ExportMetricsServiceRequest.parseFrom() method, it parses data point attributes into the data point's unknownFields instead of its labels_. Logging the output to stdout shows everything correctly (in the right places, as attributes under each data point), so please let me know where I could be going wrong. Here is the structure of the parsed request: And here is what I am (correctly) getting through the logging exporter:

I am uncertain if this is an issue with the parseFrom() method or if there is an issue with how I am configuring the Collector. Here is my configuration (with some information omitted).

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  creationTimestamp: "2023-08-22T12:38:36Z"
  generation: 109
  labels:
    app: opentelemetry
    app.kubernetes.io/managed-by: opentelemetry-operator
    component: otel-collector
  name: otel-collector
  namespace: opentelemetry-operator-system
  resourceVersion: "3069133"
  uid: <id>
spec:
  config: |
    receivers:
      prometheus:
        config:
          scrape_configs:
            - job_name: "actuator-prometheus"
              scrape_interval: 10s
              metrics_path: '/actuator/prometheus'
              kubernetes_sd_configs:
               - role: pod
              scheme: https
              tls_config:
                insecure_skip_verify: true
              relabel_configs:
               - source_labels: [__meta_kubernetes_pod_name]
                 action: keep
                 regex: <some regex>

    processors:
      batch:
      memory_limiter:
        check_interval: 1s
        limit_percentage: 50
        spike_limit_percentage: 30

    exporters:
      logging:
        verbosity: detailed
      prometheusremotewrite:
        endpoint: <prometheus endpoint>
        external_labels:
          server: otel
      otlphttp:
        metrics_endpoint: <endpoint>
        compression: none
        tls:
          insecure_skip_verify: true
        headers:
          X-Apikey: <apikey>
    service:
      pipelines:
        metrics:
          receivers: [prometheus]
          processors: [batch,memory_limiter]
          exporters: [logging,prometheusremotewrite,otlphttp]
  ingress:
    route: {}
  managementState: managed
  mode: daemonset
  observability:
    metrics: {}
  podDisruptionBudget:
    maxUnavailable: 1
  replicas: 1
  resources: {}
  targetAllocator:
    prometheusCR:
      scrapeInterval: 30s
    resources: {}
  upgradeStrategy: automatic
status:
  version: 0.83.0

[Cloud-Yao]

@hxliu33 hi, I have similar needs , and send opentelemetry data to my custom backend . As so far , I always get error as execute code , such as "ExportLogsServiceRequest.parseFrom(data)", "ExportMetricsServiceRequest.parseFrom(data)", I don't find any more information on google, this is my question detail ( https://github.com/open-telemetry/opentelemetry-proto/issues/521) ,cloud you help me ？

[hxliu33]

One thing you could change is use io.opentelemetry-proto as the groupId for the dependency instead because that was my issue (it seems they switched from io.opentelemetry to io.opentelemetry-proto at some point a few years ago).

On Sun, Jan 7, 2024 at 11:10 AM Shawn Chan @.***> wrote:

@hxliu33 https://github.com/hxliu33 hi, I have similar needs , and send opentelemetry data to my custom backend . As so far , I always get error as execute code , such as "ExportLogsServiceRequest.parseFrom(data)", "ExportMetricsServiceRequest.parseFrom(data)", I don't find any more information on google, this is my question detail ( open-telemetry/opentelemetry-proto#521 https://github.com/open-telemetry/opentelemetry-proto/issues/521) ,cloud you help me ？

— Reply to this email directly, view it on GitHub https://github.com/open-telemetry/opentelemetry-collector/issues/8710#issuecomment-1880070589, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGIQSW4MOYHPD67L3XLOK63YNKUE3AVCNFSM6AAAAAA6HP2XZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGA3TANJYHE . You are receiving this because you were mentioned.Message ID: @.***>

[Cloud-Yao]

@hxliu33 I try it , but still get error (Error Info is "Protocol message tag had invalid wire type") . Now my io.opentelemetry-proto dependency version is 1.0.0-alpha , my opentelemetry collector switch 0.83.0 version. I watch you issue , I am not sure the versions is different with your versions . I mind you use opentelemetry-operator to deploy opentelemetry collector , but I use Helm Chart to deploy opentelemetry collector (Helm Chart https://artifacthub.io/packages/helm/opentelemetry-helm/opentelemetry-collector) , Could you please provide the complete configuration you used to deploy the OpenTelemetry Collector and the full Java Web code for receiving OTLP data at that time? I'd like to use it as a reference.