search

spring-cloud / spring-cloud-gateway

An API Gateway built on Spring Framework and Spring Boot providing routing and more.
http://cloud.spring.io
Apache License 2.0
4.54k stars 3.33k forks source link

LEAK: ByteBuf.release() error #3376

Open hmorris70 opened 6 months ago

hmorris70 commented 6 months ago

We have been experiencing issues with the "LEAK: ByteBuf.release()" error. We have been addressing our shortcomings in an effort to eliminate them and while we have seen improvements we are still having an issue for which we believe is happening in the spring library.

We feel a lot of the issues we saw happened in the code where we used the ModifyRequestBodyGatewayFilterFactory or ModifyResponseBodyGatewayFilterFactory and a publisher to make changes to the request or response. In same cases we would pass through this logic needlessly. We have reduced the flow to just changes in the exchange for the request with a final filter that uses the ModifyRequestBodyGatewayFilterFactory and publisher to handle the rewrite however it appears from the logging we are still seeing memory leaks around this logic.

Most of the stack traces we are now seeing all appear to start with the same 8 lines like the following. 

#011io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:404)
#011io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
#011io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
#011io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:116)
#011org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:72)
#011org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
#011org.springframework.core.codec.CharSequenceEncoder.encodeValue(CharSequenceEncoder.java:90)
#011org.springframework.core.codec.CharSequenceEncoder.lambda$encode$0(CharSequenceEncoder.java:74)

A stack trace of a specific example of the issue follows: 

2024-04-25T17:10:48.353 - ERROR [****************************] - Trace[94f84278-3499-4b65-994a-866c86078c2e] - [reactor-http-epoll-2] - i.n.u.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
Created at:
#011io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:404)
#011io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
#011io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
#011io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:116)
#011org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:72)
#011org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
#011org.springframework.core.codec.CharSequenceEncoder.encodeValue(CharSequenceEncoder.java:90)
#011org.springframework.core.codec.CharSequenceEncoder.lambda$encode$0(CharSequenceEncoder.java:74)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:158)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(MonoContextWriteRestoringThreadLocals.java:109)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
#011reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
#011reactor.core.publisher.MonoSupplier$MonoSupplierSubscription.request(MonoSupplier.java:148)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.Operators$BaseFluxToMonoOperator.request(Operators.java:2041)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(MonoContextWriteRestoringThreadLocals.java:155)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2341)
#011reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onSubscribe(FluxOnErrorResume.java:74)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(MonoContextWriteRestoringThreadLocals.java:94)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.Operators$BaseFluxToMonoOperator.onSubscribe(Operators.java:2025)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.MonoSupplier.subscribe(MonoSupplier.java:48)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
#011reactor.core.publisher.Mono.subscribe(Mono.java:4495)
#011reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
#011reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
#011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
#011reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:122)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(MonoContextWriteRestoringThreadLocals.java:109)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
#011reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
#011reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
#011reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onComplete(FluxPeekFuseable.java:277)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
#011reactor.netty.channel.FluxReceive.terminateReceiver(FluxReceive.java:483)
#011reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:275)
#011reactor.netty.channel.FluxReceive.request(FluxReceive.java:133)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.request(FluxPeekFuseable.java:144)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.Operators$BaseFluxToMonoOperator.request(Operators.java:2041)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(MonoContextWriteRestoringThreadLocals.java:155)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.request(FluxDefaultIfEmpty.java:98)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
#011reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2341)
#011reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2215)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.Operators$BaseFluxToMonoOperator.onSubscribe(Operators.java:2025)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(MonoContextWriteRestoringThreadLocals.java:94)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.Operators$BaseFluxToMonoOperator.onSubscribe(Operators.java:2025)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onSubscribe(FluxPeekFuseable.java:178)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
#011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
#011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
#011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
#011reactor.netty.channel.FluxReceive.startReceiver(FluxReceive.java:172)
#011reactor.netty.channel.FluxReceive.lambda$subscribe$2(FluxReceive.java:150)
#011io.opentelemetry.javaagent.bootstrap.executors.ContextPropagatingRunnable.run(ContextPropagatingRunnable.java:37)
#011io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
#011io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
#011io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
#011io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)
#011io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
#011io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
#011io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
#011java.base/java.lang.Thread.run(Thread.java:840)

This is a portion of the logging related to the trace id on the above stack trace and is limited to the ModifyRequestBodyFromExchangeFilter and "LEAK: ByteBuf.release()" error that is happening in the middle of the filter indicating to us that it seems to be happening while the spring code is running which is consistent with those first 8 lines of the stack trace. 

2024-04-25T17:10:48.348 - DEBUG [****************************] - Trace[94f84278-3499-4b65-994a-866c86078c2e] - [reactor-http-epoll-2] - c.p.g.P.f.ModifyRequestBodyFromExchangeFilter - Running ModifyRequestBodyFromExchangeGatewayFilterFactory
2024-04-25T17:10:48.353 - ERROR [****************************] - Trace[94f84278-3499-4b65-994a-866c86078c2e] - [reactor-http-epoll-2] - i.n.u.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
2024-04-25T17:10:48.353 - DEBUG [****************************] - Trace[94f84278-3499-4b65-994a-866c86078c2e] - [reactor-http-epoll-2] - c.p.g.P.f.ModifyRequestBodyFromExchangeFilter - originalRequestBody is null false, modifiedRequestBody is null false
2024-04-25T17:10:48.353 - DEBUG [****************************] - Trace[94f84278-3499-4b65-994a-866c86078c2e] - [reactor-http-epoll-2] - c.p.g.P.f.ModifyRequestBodyFromExchangeFilter - originalRequestBody: <SERVICE execution_audit_id="540968606" request_type="CartAbandonmentRecoveryTask" session_id="vrll.131048.297"/>

This is the filter is, the debug logging was added when we started to observe memory leaks after the rewrite. 

package com.platform.gateway.PlatformSpringCloudGateway.filter;

import static com.platform.gateway.PlatformSpringCloudGateway.utils.Constants.CACHED_REQUEST_BODY;
import static com.platform.gateway.PlatformSpringCloudGateway.utils.Constants.CACHED_REQUEST_BODY_MODIFIED;

import org.reactivestreams.Publisher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.gateway.filter.GatewayFilter;
import org.springframework.cloud.gateway.filter.factory.AbstractGatewayFilterFactory;
import org.springframework.cloud.gateway.filter.factory.rewrite.ModifyRequestBodyGatewayFilterFactory;
import org.springframework.cloud.gateway.filter.factory.rewrite.RewriteFunction;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;

import lombok.extern.slf4j.Slf4j;
import reactor.core.publisher.Mono;

@Component
@Slf4j
public class ModifyRequestBodyFromExchangeFilter
    extends AbstractGatewayFilterFactory<ModifyRequestBodyFromExchangeFilter.Config> {
  private ModifyRequestBodyGatewayFilterFactory modifyRequestBodyGatewayFilterFactory;

  @Autowired
  public ModifyRequestBodyFromExchangeFilter(
      ModifyRequestBodyGatewayFilterFactory modifyRequestBodyGatewayFilterFactory) {
    super(Config.class);
    this.modifyRequestBodyGatewayFilterFactory = modifyRequestBodyGatewayFilterFactory;
  }

  @Override
  public GatewayFilter apply(final Config config) {
    return (exchange, chain) -> {
      if (Boolean.TRUE.equals(exchange.getAttribute(CACHED_REQUEST_BODY_MODIFIED))) {
        log.debug("Running ModifyRequestBodyFromExchangeGatewayFilterFactory");
        String modifiedRequestBody = exchange.getAttribute(CACHED_REQUEST_BODY);
        return modifyRequestBodyGatewayFilterFactory
            .apply(
                c ->
                    c.setRewriteFunction(
                        String.class, String.class, new RequestBodyEnhancer(modifiedRequestBody)))
            .filter(exchange, chain);
      }

      return chain.filter(exchange);
    };
  }

  public static class RequestBodyEnhancer implements RewriteFunction<String, String> {
    private String modifiedRequestBody;

    public RequestBodyEnhancer(String modifiedRequestBody) {
      this.modifiedRequestBody = modifiedRequestBody;
    }

    @Override
    public Publisher<String> apply(ServerWebExchange exchange, String originalRequestBody) {
      log.debug(
          "originalRequestBody is null {}, modifiedRequestBody is null {}",
          (originalRequestBody == null),
          (modifiedRequestBody == null));
      if (originalRequestBody != null) {
        log.debug(
            "originalRequestBody: {}",
            originalRequestBody.substring(
                0, originalRequestBody.length() > 500 ? 500 : originalRequestBody.length()));
      }

      if (originalRequestBody == null) {
        // they should not be calling me with a null and if they do it seems like something is wrong.
        return Mono.empty();
      }

      exchange
          .getAttributes()
          .put(CACHED_REQUEST_BODY_MODIFIED, false); // reset the flag when writing the body
      return Mono.just(modifiedRequestBody);
    }
  }

  public static class Config {}
}

Java version: 17 Spring boot version: 3.1.9 Spring cloud version: 2022.0.5 JVM args passed: -XX:MaxRAMPercentage=80.0 -Dio.netty.leakDetection.level=PARANOID

spencergibb commented 6 months ago

Have you tried with the latest 2023.0.1? There have been a number of fixes in this area.

hmorris70 commented 6 months ago

We have not upgrade beyond the current versions as we are blocked from doing so until we rewrite the logic that uses the WebFluxTags

spencergibb commented 6 months ago

Let me know when you get a chance.

hmorris70 commented 6 months ago

Thank you Spencer Gibb we have prioritized that work and should be able to update soon if there are still any issues remaining.

spring-cloud-issues commented 6 months ago

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

spring-cloud-issues commented 6 months ago

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

wangmengpro commented 5 months ago

Have you tried with the latest 2023.0.1? There have been a number of fixes in this area.

Is there any other solution if I don't upgrade? The cost of upgrading is too high

hmorris70 commented 5 months ago

I apologize for the delay in feedback I expected that my coworkers would follow up on this when we finally reproduced the issue. We have updated our development environment to the following:

Java version: 17 Spring boot version: 3.2.5 Spring cloud version: 2023.0.1 JVM args passed: -XX:MaxRAMPercentage=80.0 -Dio.netty.leakDetection.level=PARANOID

Since this is our development environment it does not receive the same traffic and we have only seen the leak detection once with the upgraded version, it still appears to start with the same 8 lines from above. I am going to continue to monitor this for more occurrences but here is the full stack trace to the one I have. 

ecs-pscg-public-dev-na3[4384]: Recent access records:
ecs-pscg-public-dev-na3[4384]: Created at:
ecs-pscg-public-dev-na3[4384]: #011io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:404)
ecs-pscg-public-dev-na3[4384]: #011io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
ecs-pscg-public-dev-na3[4384]: #011io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
ecs-pscg-public-dev-na3[4384]: #011io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:116)
ecs-pscg-public-dev-na3[4384]: #011org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:72)
ecs-pscg-public-dev-na3[4384]: #011org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
ecs-pscg-public-dev-na3[4384]: #011org.springframework.core.codec.CharSequenceEncoder.encodeValue(CharSequenceEncoder.java:90)
ecs-pscg-public-dev-na3[4384]: #011org.springframework.core.codec.CharSequenceEncoder.lambda$encode$0(CharSequenceEncoder.java:74)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:283)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:158)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(MonoContextWriteRestoringThreadLocals.java:110)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2097)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onComplete(FluxPeekFuseable.java:277)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(MonoContextWriteRestoringThreadLocals.java:111)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2097)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(MonoContextWriteRestoringThreadLocals.java:111)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(MonoContextWriteRestoringThreadLocals.java:156)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.request(Operators.java:2067)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(MonoContextWriteRestoringThreadLocals.java:156)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.request(FluxPeekFuseable.java:144)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.request(Operators.java:2067)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(MonoContextWriteRestoringThreadLocals.java:156)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2241)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(MonoContextWriteRestoringThreadLocals.java:95)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.onSubscribe(Operators.java:2051)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onSubscribe(FluxPeekFuseable.java:178)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(MonoContextWriteRestoringThreadLocals.java:95)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.onSubscribe(Operators.java:2051)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(MonoContextWriteRestoringThreadLocals.java:95)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Mono.subscribe(Mono.java:4568)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:265)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:241)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:204)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(MonoContextWriteRestoringThreadLocals.java:142)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:155)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2842)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onComplete(FluxMapFuseable.java:350)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:85)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:147)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:159)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(MonoContextWriteRestoringThreadLocals.java:110)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2097)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onComplete(FluxContextWriteRestoringThreadLocals.java:149)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onComplete(TracingSubscriber.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onComplete(FluxOnAssembly.java:549)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2573)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.request(Operators.java:2067)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(MonoContextWriteRestoringThreadLocals.java:156)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.request(FluxHide.java:152)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.request(FluxContextWriteRestoringThreadLocals.java:163)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2241)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(MonoContextWriteRestoringThreadLocals.java:95)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onSubscribe(FluxHide.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Operators$BaseFluxToMonoOperator.onSubscribe(Operators.java:2051)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onSubscribe(FluxContextWriteRestoringThreadLocals.java:104)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onSubscribe$0(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onSubscribe(TracingSubscriber.java:59)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals.subscribe(MonoContextWriteRestoringThreadLocals.java:44)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.Mono.subscribe(Mono.java:4568)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:265)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxUsingWhen$UsingWhenSubscriber.onNext(FluxUsingWhen.java:348)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onNext(MonoFlatMapMany.java:251)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.innerNext(FluxConcatMapNoPrefetch.java:259)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:865)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:122)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$FuseableContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:190)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:245)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:97)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
ecs-pscg-public-dev-na3[4384]: #011io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(MonoContextWriteRestoringThreadLocals.java:110)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
ecs-pscg-public-dev-na3[4384]: #011reactor.core.publisher.FluxContextWriteRestoringThreadLocals$ContextWriteRestoringThreadLocalsSubscriber.onNext(FluxContextWriteRestoringThreadLocals.java:118)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.RedisPublisher$ImmediateSubscriber.onNext(RedisPublisher.java:890)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.RedisPublisher$RedisSubscription.onNext(RedisPublisher.java:291)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.RedisPublisher$SubscriptionCommand.doOnComplete(RedisPublisher.java:777)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.protocol.CommandWrapper.complete(CommandWrapper.java:65)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.protocol.CommandWrapper.complete(CommandWrapper.java:63)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.protocol.CommandHandler.complete(CommandHandler.java:745)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.protocol.CommandHandler.decode(CommandHandler.java:680)
ecs-pscg-public-dev-na3[4384]: #011io.lettuce.core.protocol.CommandHandler.channelRead(CommandHandler.java:597)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:801)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
ecs-pscg-public-dev-na3[4384]: #011io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
ecs-pscg-public-dev-na3[4384]: #011io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
ecs-pscg-public-dev-na3[4384]: #011io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
ecs-pscg-public-dev-na3[4384]: #011io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
ecs-pscg-public-dev-na3[4384]: #011java.base/java.lang.Thread.run(Thread.java:840)
1120562641 commented 5 months ago

append: gradle dependency id 'org.springframework.boot' version '3.2.6' id 'io.spring.dependency-management' version '1.1.5' org.springframework.cloud:spring-cloud-dependencies:2023.0.2

I tried using the above version, which seems to have fixed the related issues. I still received a leak report, but my production environment memory usage did not continue to increase as before. Maybe you can try the latest version.

hmorris70 commented 5 months ago

@spencergibb or @spring-cloud-issues can we reopen this? I provided stack trace from a more recent issue with the noted spring versions. I also have a few more now. We are testing with the latest spring version in a development environment only.

spencergibb commented 1 month ago

@hmorris70 have you tried the versions mentioned in https://github.com/spring-cloud/spring-cloud-gateway/issues/3376#issuecomment-2167047974

hmorris70 commented 1 month ago

@spencergibb yes we have updated here are the current versions

id 'org.springframework.boot' version '3.2.6' id 'io.spring.dependency-management' version '1.1.5'

dependencyManagement { imports { mavenBom "org.springframework.cloud:spring-cloud-dependencies:2023.0.2" } // Override the versions listed in the BOM dependencies { dependency 'org.springframework.cloud:spring-cloud-starter-gateway:4.1.3' } }

We are still seeing a slight memory leak in some prod instances, unfourtunely reproduction in lower environments with more details is difficult. We did just recently have a couple of the LEAK: ByteBuf.release() errors in prod so I can pull those stack traces and provide them in case it will help.

hmorris70 commented 1 month ago

Here are the three stack traces I mentioned in my previous post. 2024-09-26 stacktraces.txt