[BUG] Cosmos SDK timeout exceptions missing the Open Telemetry trace details

[Neha-Sharma0-WM]

Describe the bug Cosmos SDK timeout exception traces missing the Open Telemetry trace details. Probable similar issue - GA OpenTelemetry tracing plugin · Issue #30557 · Azure/azure-sdk-for-java (github.com).

Exception or Stack Trace Add the exception log and stack trace if available

To Reproduce

• Setup Cosmos SDK Version 4.34.0
• Replicate a timeout/error from DB so that SDK throws an exception
• We use wrapper implementation of below Opentelemetry API version to instrument our code. We also have JaegerThriftUDP and OTLP endpoints available on the Opentelemetry Agent (i.e. OpentelemetryCollector Instance running on the host).

<!-- [https://mvnrepository.com/artifact/io.opentelemetry/opentelemetry-api](https://urldefense.com/v3/__https://mvnrepository.com/artifact/io.opentelemetry/opentelemetry-api__;!!IfjTnhH9!VGe8BgXkoxnLr5X9_50iQmw_GrcKgU9HHIfe2W8s-ZHmej-A7hx1cpXVsHg6gS3U7lRUCL7TwAa1tWrS2bhBa6mgLreQhA$) -->
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-api</artifactId>
    <version>1.15.0</version>
</dependency>

• The exception trace is missing the traceparent/traceID details

Code Snippet Here's how we're using SDK. We have a soft timeout of 600 ms on the read call (using block) i.e. if no response from SDK within 600ms, we force timeout -

CosmosItemResponse<POJO> response = CosmosAsyncContainer.readItem(itemId, new PartitionKey(<requestKey>), POJO.class)
                        .block(Duration.ofMillis(ccmService.getCosmosConfig().getRequestTimeOut()));

Expected behavior The exception from SDK should have trace ID.

Screenshots If applicable, add screenshots to help explain your problem.

Setup (please complete the following information):

• OS: Linux
• IDE: Intellij
• Library/Libraries: azure-cosmos:jar:4.34.0
• Java version: Azul 17
• App Server/Environment: Tomcat, Apache CXF
• Frameworks: Spring, REST APIs

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:

• verbose dependency tree (mvn dependency:tree -Dverbose)
• exception message, full stack trace, and any available logs

Additional context Add any other context about the problem here.

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [ ] Bug Description Added
• [ ] Repro Steps Added
• [ ] Setup information Added

[lmolkova]

@Neha-Sharma0-WM, thank you for reporting this!

We're looking at it, but need the following information:

• can you please share if CosmosDb is used in Direct or Gateway mode? It has a huge effect on tracing.
• can you please elaborate on "The exception from SDK should have trace ID." ? Could you please share a screenshot of the trace you see and explain what you'd like to have there?
  • Do you expect to see trace-id in exception message?
  • Or do you expect to see exception log/event under the CosmosDB span and don't see it?

Thank you!

[lmolkova]

/cc @simplynaveen20

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @kushagraThapar, @TheovanKraay

[Neha-Sharma0-WM]

Hey @lmolkova,

Thanks for taking a look at this. Here are the details you requested -

• We're using comosDB in direct mode.
• We expect to see traceId in the exception message from SDK. We debug our issues using traceId and currently because these exceptions do not get attached to our transaction journey logs (due to missing trace context), we tend to lose them during our debug sessions/investigation. One example of such SDK exceptopn would be GoneException.

Hope this helps, thanks

[kushagraThapar]

cc @FabianMeiswinkel / @sourabh1007

[kushagraThapar]

This has been done.