open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.92k stars 2.28k forks source link

Should X-Ray Trace exporter not create Segment for Consumer SpanKind? #20267

Closed krissrex closed 1 year ago

krissrex commented 1 year ago

Component(s)

exporter/awsxray

What happened?

Description

The current AWS X-Ray exporter will create subsegments of all spans but Server. For an application using messaging, like an AWS SQS consumer, it seems correct to use SpanKind Consumer https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md#spankind .

I try to connect the consumer to a parent using the io.opentelemetry.contrib:opentelemetry-aws-xray-propagator:1.23.0-alpha like this and span.setParent.

However, X-Ray will see these as sub-segments, instead of remotes. Is this correct? Is producer-consumer only intended to be internally in an application, not for applications communicating events via SQS?

Steps to Reproduce

  1. Use otel-collector with xray
  2. Use ECS application with SQS. SQS raw message delivery. Attributes ask for AWSTraceHeader.
  3. Create spans after SQS receiveMessage:
    tracer.spanBuilder("myEvent receive")
          .apply { createSpanFromSqs(message)?.let { this.setParent(it) } }
          .setSpanKind(SpanKind.CONSUMER)
    // Then startSpan, makeCurrent, end etc.

where I get a parent like this:

fun createSpanFromSqs(message: Message): Context? {
  val xrayTracingId = message.attributes().get(MessageSystemAttributeName.AWS_TRACE_HEADER)

  if (xrayTracingId == null) {
    return null
  }

  val context: Context =
      AwsXrayPropagator.getInstance()
          .extract(Context.root(), mapOf("X-Amzn-Trace-Id" to xrayTracingId), getter)

  return context
}

Expected Result

Segments in xray called myEvent receive.

Actual Result

Subsegment nested under some other service, making it look internal.

Collector version

0.26.1

Environment information

Environment

"otel.resource.cloud.platform": "aws_ecs",
"otel.resource.os.type": "linux",
"otel.resource.aws.ecs.launchtype": "fargate",
"otel.resource.host.arch": "amd64",
"otel.resource.telemetry.auto.version": "1.23.0-aws",
"otel.resource.os.description": "Linux 5.10.167-147.601.amzn2.x86_64",
"otel.resource.process.runtime.description": "Azul Systems, Inc. OpenJDK 64-Bit Server VM 19.0.2+7",
"otel.resource.process.runtime.version": "19.0.2+7",

OpenTelemetry Collector configuration

extensions:
  health_check:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:55681
  awsxray:
    endpoint: 0.0.0.0:2000
    transport: udp
  statsd:
    endpoint: 0.0.0.0:8125
    aggregation_interval: 60s
  awsecscontainermetrics:

processors:
  batch/traces:
    timeout: 1s
    send_batch_size: 50

exporters:
  awsxray:
    indexed_attributes: ["otel.resource.service.name"] # Max 50 attributes

service:
  pipelines:
    traces:
      receivers: [otlp,awsxray]
      processors: [batch/traces]
      exporters: [awsxray]

  extensions: [health_check]

Log output

No response

Additional context

No response

github-actions[bot] commented 1 year ago

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

willarmiros commented 1 year ago

Hi @krissrex - thanks for raising this. This issue has been reported here as well: https://github.com/aws-observability/aws-otel-collector/issues/1773

We will be taking a look at fixing this as soon as possible and post an update when we can.

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 year ago

This issue has been closed as inactive because it has been stale for 120 days with no activity.