search

craigatk / opentelemetry-gradle-plugin

Instrument your Gradle build with OpenTelemetry observability to find and eliminate bottlenecks in your build
Apache License 2.0
30 stars 9 forks source link

Connection reset by peer #169

Open mleegwt opened 4 days ago

mleegwt commented 4 days ago

I've followed https://github.com/craigatk/opentelemetry-gradle-plugin#jaeger to setup Jaeger to receive traces. When enabling this plugin on a (quite small) internal project using Gradle 8.8 and with or without parallel mode for execution I consistently encounter the following stacktrace (that does not occur without this plugin):

Unexpected exception thrown.
org.gradle.internal.remote.internal.MessageIOException: Could not write '/127.0.0.1:61099'.
        at org.gradle.internal.remote.internal.inet.SocketConnection.flush(SocketConnection.java:140)
        at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionDispatch.run(MessageHub.java:333)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:47)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.io.IOException: Connection reset by peer
        at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method)
        at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:54)
        at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:132)
        at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:76)
        at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:53)
        at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:532)
        at org.gradle.internal.remote.internal.inet.SocketConnection$SocketOutputStream.writeWithNonBlockingRetry(SocketConnection.java:279)
        at org.gradle.internal.remote.internal.inet.SocketConnection$SocketOutputStream.writeBufferToChannel(SocketConnection.java:267)
        at org.gradle.internal.remote.internal.inet.SocketConnection$SocketOutputStream.flush(SocketConnection.java:261)
        at org.gradle.internal.remote.internal.inet.SocketConnection.flush(SocketConnection.java:138)
        ... 6 more

This also fails the build! This is the most annoying part of this issue. I don't seen any pointers towards anything specific apart from that it consistently happens during test execution. Some of these tests are kind of slow (approximately 20 seconds, sometimes more). Could that have anything to do with it?

mleegwt commented 4 days ago

A new test seems to indicate that it's not as simple as that:

    @Test
    void testIssue() throws InterruptedException {
        Thread.sleep(20_000);
    }