Different topic names for Metrics,Logs and Traces instead of default topic names

[dharmaplay]

Would like to send the telemetry data to different configured topics instead of default topic names like default = otlp_spans for traces, otlp_metrics for metrics, otlp_logs for logs, Could you help me to send the telemetry data to configured topic names.

Thanks Dharma

[jpkrohling]

Is this about the Kafka exporter?

[dharmaplay]

Yes

[jpkrohling]

cc @pavolloffay @MovieStoreGuy

[pavolloffay]

hi @dharmaplay, did you read the configuration and tried to set the topic name?

https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/kafkaexporter#kafka-exporter

[dharmaplay]

Hi @pavolloffay , Yes gone through, but If I want to send Metrics , Logs and traces to different topics, then don't have parameters to separate them,Example, If I want the Metrics to go Metrics-Test kafka topic , Logs to Logs-Test kafka topic and Traces to Taces-Test kafka topic, so this type of capability it does not contain. so Could you help me is my understanding is right?

[pavolloffay]

I would like to hear your use case as well, why you want to override the default name.

You could achieve your requirement by defining multiple (for metrics and logs) kafka exporters and use them for in pipelines. Something like this should work:

exporters:
  kafka/logs:
    topic: logs-test
  kafka/metrics:
    topic: metrics-test
    ​
service:
  pipelines:
    logs:
      exporters: [kafka/logs]
    metrics:
      exporters: [kafka/metrics]

[dharmaplay]

We have some standards to create topic names and already these topic names are being used by some other team, So wanted to create different topics names for each of these Metrics , Logs and Traces. I tried below configuration but started throwing below error employee-service_1 | [otel.javaagent 2022-02-07 15:10:32:244 +0000] [OkHttp http://otel-collector:4317/...] ERROR io.opentelemetry.exporter.otlp.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: otel-collector: Name or service not known My configuration looks below

receivers:
  otlp:
    protocols:
      grpc:

exporters:
  kafka/metrics:
    brokers: "broker:9092"
    topic: "test.metrics"
    encoding: "otlp_json"
    protocol_version: 2.0.0
  kafka/logs:
    brokers: "broker:9092"
    topic: "test.logs"
    encoding: "otlp_json"
    protocol_version: 2.0.0  
  kafka/spans:
    brokers: "broker:9092"
    topic: "test.spans"
    encoding: "otlp_json"
    protocol_version: 2.0.0      
processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679

service:
  extensions: [pprof, zpages, `health_check]`
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [kafka/metrics]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [kafka/logs]
    spans:
      receivers: [otlp]
      processors: [batch]
      exporters: [kafka/spans]

[pavolloffay]

employee-service_1 | [otel.javaagent 2022-02-07 15:10:32:244 +0000] [OkHttp http://otel-collector:4317/...] ERROR io.opentelemetry.exporter.otlp.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: otel-collector: Name or service not known

The error message seems from the Java agent. Make sure the java agent can talk to otel-collector deployment.

[dharmaplay]

If I configure the collector in the below manner by specifying single topic then I am getting below json ( Not sure is Agent is sending all 3 componenets data to Single Topic ) but If I configure separate topics for each SPANS, LOGS and METRICS a separate topics then I will get above error.

receivers:
  otlp:
    protocols:
      grpc:

exporters:
  kafka:
    brokers: "broker:9092"  
    topic: "test.topic"
    encoding: "otlp_json"
    protocol_version: 2.0.0

  logging:

  zipkin:
    endpoint: "http://zipkin:9411/api/v2/spans"
    format: proto

  jaeger:
    endpoint: jaeger:14250
    tls:
      insecure: true

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679

service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [kafka, logging, zipkin, jaeger]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]

` JSON MESSAGE:

{
    "resourceSpans": [{
        "resource": {
            "attributes": [{
                "key": "host.arch",
                "value": {
                    "stringValue": "amd64"
                }
            }, {
                "key": "host.name",
                "value": {
                    "stringValue": "99e9ac97c415"
                }
            }, {
                "key": "os.description",
                "value": {
                    "stringValue": "Linux 5.10.76-linuxkit"
                }
            }, {
                "key": "os.type",
                "value": {
                    "stringValue": "linux"
                }
            }, {
                "key": "process.command_line",
                "value": {
                    "stringValue": "/usr/lib/jvm/java-11-openjdk-11.0.4.11-1.el7_7.x86_64:bin:java -javaagent:./opentelemetry-javaagent.jar -Dotel.resource.attributes=service.name=employee-service"
                }
            }, {
                "key": "process.executable.path",
                "value": {
                    "stringValue": "/usr/lib/jvm/java-11-openjdk-11.0.4.11-1.el7_7.x86_64:bin:java"
                }
            }, {
                "key": "process.pid",
                "value": {
                    "intValue": "1"
                }
            }, {
                "key": "process.runtime.description",
                "value": {
                    "stringValue": "Oracle Corporation OpenJDK 64-Bit Server VM 11.0.4+11-LTS"
                }
            }, {
                "key": "process.runtime.name",
                "value": {
                    "stringValue": "OpenJDK Runtime Environment"
                }
            }, {
                "key": "process.runtime.version",
                "value": {
                    "stringValue": "11.0.4+11-LTS"
                }
            }, {
                "key": "service.name",
                "value": {
                    "stringValue": "employee-service"
                }
            }, {
                "key": "telemetry.auto.version",
                "value": {
                    "stringValue": "1.10.0"
                }
            }, {
                "key": "telemetry.sdk.language",
                "value": {
                    "stringValue": "java"
                }
            }, {
                "key": "telemetry.sdk.name",
                "value": {
                    "stringValue": "opentelemetry"
                }
            }, {
                "key": "telemetry.sdk.version",
                "value": {
                    "stringValue": "1.10.0"
                }
            }]
        },
        "instrumentationLibrarySpans": [{
            "instrumentationLibrary": {
                "name": "io.opentelemetry.spring-webmvc-3.1",
                "version": "1.10.0"
            },
            "spans": [{
                "traceId": "8c8f278ab417f9a709979897c54e7d35",
                "spanId": "bc071d7a09240fc7",
                "parentSpanId": "f13b4555f65a2aa6",
                "name": "EmployeeController.findByOrganization",
                "kind": "SPAN_KIND_INTERNAL",
                "startTimeUnixNano": "1643619245293964600",
                "endTimeUnixNano": "1643619245302618100",
                "attributes": [{
                    "key": "thread.name",
                    "value": {
                        "stringValue": "http-nio-8080-exec-3"
                    }
                }, {
                    "key": "thread.id",
                    "value": {
                        "intValue": "31"
                    }
                }],
                "status": {}
            }]
        }, {
            "instrumentationLibrary": {
                "name": "io.opentelemetry.tomcat-7.0",
                "version": "1.10.0"
            },
            "spans": [{
                "traceId": "8c8f278ab417f9a709979897c54e7d35",
                "spanId": "f13b4555f65a2aa6",
                "parentSpanId": "",
                "name": "/employee/organization/{organizationId}",
                "kind": "SPAN_KIND_SERVER",
                "startTimeUnixNano": "1643619245289990000",
                "endTimeUnixNano": "1643619245303787700",
                "attributes": [{
                    "key": "net.peer.port",
                    "value": {
                        "intValue": "63064"
                    }
                }, {
                    "key": "http.host",
                    "value": {
                        "stringValue": "localhost:8080"
                    }
                }, {
                    "key": "net.peer.ip",
                    "value": {
                        "stringValue": "172.29.0.1"
                    }
                }, {
                    "key": "thread.name",
                    "value": {
                        "stringValue": "http-nio-8080-exec-3"
                    }
                }, {
                    "key": "net.transport",
                    "value": {
                        "stringValue": "ip_tcp"
                    }
                }, {
                    "key": "http.flavor",
                    "value": {
                        "stringValue": "1.1"
                    }
                }, {
                    "key": "http.target",
                    "value": {
                        "stringValue": "/employee/organization/1"
                    }
                }, {
                    "key": "http.scheme",
                    "value": {
                        "stringValue": "http"
                    }
                }, {
                    "key": "http.status_code",
                    "value": {
                        "intValue": "200"
                    }
                }, {
                    "key": "http.method",
                    "value": {
                        "stringValue": "GET"
                    }
                }, {
                    "key": "http.server_name",
                    "value": {
                        "stringValue": "localhost"
                    }
                }, {
                    "key": "thread.id",
                    "value": {
                        "intValue": "31"
                    }
                }, {
                    "key": "net.peer.name",
                    "value": {
                        "stringValue": "gateway"
                    }
                }, {
                    "key": "http.user_agent",
                    "value": {
                        "stringValue": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0"
                    }
                }],
                "status": {}
            }]
        }],
        "schemaUrl": "https://opentelemetry.io/schemas/1.8.0"
    }]
}

[MovieStoreGuy]

Hey @dharmaplay ,

looking at this specific error here:

employee-service_1 | [otel.javaagent 2022-02-07 15:10:32:244 +0000] [OkHttp http://otel-collector:4317/...] ERROR io.opentelemetry.exporter.otlp.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: otel-collector: Name or service not known

It looks like the otel-collector wasn't available for your java client to communicate with? I suspect this potentially might be a start up ordering error if you're running in something like docker-compose or kube.

Is this still an issue that is impacting you as well?

[github-actions[bot]]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[jpkrohling]

Closing this, as this doesn't seem relevant anymore.