Internal traces generate recursively

[danelson]

Describe the bug I have some interest in consuming the collector internal traces. I have started experimenting with them and they are seemingly generated recursively.

Steps to reproduce Use configuration below. Send a single log to the collector. Observe that otelcol_receiver_accepted_spans increases indefinitely as well as the number of exported traces.

What did you expect to see? I expect internal traces to not generate additional traces.

What did you see instead? Trace count increases forever.

What version did you use? v0.104.0

What config did you use?

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
        cors:
          allowed_origins: ["*"]
          allowed_headers: ["*"]

exporters:
  debug:
    verbosity: detailed
  otlp/jaeger:
    endpoint: http://jaeger:4317
    tls:
      insecure: true

extensions:
  zpages:
    endpoint: 0.0.0.0:55679

service:
  telemetry:
    logs:
      level: info
      encoding: console
    metrics:
      level: detailed
    traces:
      processors:
        - batch:
            exporter:
              otlp:
                protocol: grpc/protobuf
                endpoint: http://localhost:4317
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [debug]
    traces:
      receivers: [otlp]
      exporters: [otlp/jaeger]

Environment docker

Additional context OpenTelemetry .NET has a concept of SuppressInstrumentationScope to help avoid what I consider a similar issue.

This seems a bit different tho because presumable the trace receiver cannot differentiate between internal and external traffic since it comes in the same way (at least how I understand it).

[yurishkuro]

duplicate of https://github.com/open-telemetry/opentelemetry-collector/issues/10663

[codeboten]

This is similar to #10663 in the sense that we're talking about filtering out unwanted spans, but in this case the source of the span is the traffic being sent directly back to the collector which is causing a new trace to be emitted about the received span. I don't know if the same solution (turn off spans for a receiver) would address this use-case, because we would still want to produce spans from incoming requests to the OTLP receiver, just not from requests the collector sent itself?

A way this could be solved is by not recording spans for operations that include the current Collector's instance ID in the resource attributes.