search

open-telemetry / opentelemetry-java-contrib

https://opentelemetry.io
Apache License 2.0
162 stars 129 forks source link

AWS Propagator throws exception when used with Netty #1034

Closed yurybubnov closed 1 month ago

yurybubnov commented 1 year ago

Component(s)

aws-xray-propagator

What happened?

Description

AWS XRay propagator sometimes fails to parse X-Amzn-Trace-Id with the following exception. Too short trace ids should return invalid trace. This might be happening because of reactive nature of the Spring Cloud Gateway - the agent starts acting on partially received data.

"java.lang.StringIndexOutOfBoundsException: begin 11, end 35, length 34
    at java.base/java.lang.String.checkBoundsBeginEnd(String.java:4606)
    at java.base/java.lang.String.substring(String.java:2709)
    at io.opentelemetry.contrib.awsxray.propagator.AwsXrayPropagator.parseShortTraceId(AwsXrayPropagator.java:298)
    at io.opentelemetry.contrib.awsxray.propagator.AwsXrayPropagator.parseTraceId(AwsXrayPropagator.java:253)
    at io.opentelemetry.contrib.awsxray.propagator.AwsXrayPropagator.getContextFromHeader(AwsXrayPropagator.java:208)
    at io.opentelemetry.contrib.awsxray.propagator.AwsXrayPropagator.extract(AwsXrayPropagator.java:169)
    at io.opentelemetry.javaagent.shaded.io.opentelemetry.context.propagation.MultiTextMapPropagator.extract(MultiTextMapPropagator.java:65)
    at io.opentelemetry.javaagent.shaded.instrumentation.api.instrumenter.PropagatingFromUpstreamInstrumenter.start(PropagatingFromUpstreamInstrumenter.java:30)
    at io.opentelemetry.javaagent.shaded.instrumentation.netty.v4_1.internal.server.HttpServerRequestTracingHandler.channelRead(HttpServerRequestTracingHandler.java:61)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
    at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509)
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:833)

Steps to Reproduce

Environment: Spring Boot 3 reactive - Spring Cloud Gateway. Opentelemtry agent used with the application via -javaagent JVM parameter Incoming requests have X-Amzn-Trace-Id headers set.

Expected Result

Incoming information from the header is parsed, and SpanContext is successfully populated.

Actual Result

Exception is thrown sometimes.

Component version

1.29.0

Log output

No response

Additional context

No response

trask commented 1 month ago

Fixed by #1036