Trace id for sqs messages is not propagated

aarrsseni commented 3 days ago

Describe the bug

I have service which is publishing sqs message to the queue and also is reading the message from this queue. It's using javaagent and have opentelemetry-aws-sdk-2.2-autoconfigure dependency. Parameters described here https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/aws-sdk/README.md are enabled as well.

I see that trace id is not propagated and there is no trace and span id at all after message consuming.

Steps to reproduce

Spring service publishes simple sqs message to the queue using spring-cloud-aws-sqs SqsTemplate. Consume the message using @SqsListener from spring-cloud-aws-sqs. Check the trace id before and after sending.

Expected behavior

Trace id and span id should be propagated and shouldn't be lost.

Actual behavior

Trace id and span id are lost after sending sqs message.

Javaagent or library instrumentation version

2.5.0

Environment

JDK: Temurin 17.0.7 OS: Alpine Linux Spring: 6.1.10 opentelemetry-aws-sdk-2.2-autoconfigure: 2.7.0-alpha spring-cloud-aws-sqs: 3.1.1

Additional context

No response

laurit commented 3 days ago

Since camel provides its own open telemetry integration you should seek assistance from the camel community. If you are able to reproduce this without using camel then please provide a minimal application that reproduces the issue along with any necessary instructions.

aarrsseni commented 2 days ago

@laurit, thanks for the reply about camel. I checked another service I have, which is not using Camel at all. I updated task description with the new case. It's using spring-cloud-aws-sqs, do you know, has open telemetry support for it? Should I configure something else? In any way for now looks like there is not enough just to add one dependency and enable few flags to propagate trace id.

laurit commented 2 days ago

It's using javaagent and have opentelemetry-aws-sdk-2.2-autoconfigure dependency.

When using java agent you don't need to include opentelemetry-aws-sdk-2.2-autoconfigure.

Spring service publishes simple sqs message to the queue using spring-cloud-aws-sqs SqsTemplate. Consume the message using @SqsListener from spring-cloud-aws-sqs. Check the trace id before and after sending.

Please provide minimal application that reproduces the issue along with any instructions necessary.

aarrsseni commented 2 days ago

@laurit, here is an example https://github.com/aarrsseni/demo-sqs-opentelemetry/tree/master/src/main. I checked in my env and I had trace id for rest request but no trace id for sqs. As I mentioned I'm using javaagent 2.5.0. Now I haven't included any opentelemetry dependencies, only javaagent.

laurit commented 1 day ago

@aarrsseni your configuration https://github.com/aarrsseni/demo-sqs-opentelemetry/blob/73a79bbcb059f70ee244fc551d8b2ef642c20eeb/src/main/resources/application.yml#L9 sets endpoint to http://docker:4566 is that localstack? If it is localstack then as far as I know localstack propagates aws trace headers only in commercial version. Idk if that has changed. If that is the case see https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/aws-sdk/aws-sdk-2.2/library/README.md#trace-propagation there is an experimental option to use a different propagation mechanism. If it is not localstack then please elaborate what it is.

aarrsseni commented 1 day ago

@laurit, yes, this is LocalStack in the example but I have the same configuration in the real env without LocalStack. I've checked parameters described in the documentation as well but it didn't help.

Maybe you have an example of working solution to check the config or some more detailed documentation?

laurit commented 23 hours ago

This will require instrumentation for @SqsListener. There is also https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/4788

open-telemetry / opentelemetry-java-instrumentation