hgomez / influxdb

InfluxDB Tools
19 stars 2 forks source link

Increase connection-, read- and write-timeouts to reasonable defaults #7

Closed amotl closed 3 years ago

amotl commented 3 years ago

Dear Henri,

after slightly improving InfluxDB Fetcher the other day, I now had an actual use case for it. I tried to export 368M / 1.5 million records worth of data into a lineprotocol file but the program croaked after five seconds on a timeout.

java.net.SocketTimeoutException ```java Exception in thread "main" org.influxdb.InfluxDBIOException: java.net.SocketTimeoutException: timeout at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:841) at org.influxdb.impl.InfluxDBImpl.executeQuery(InfluxDBImpl.java:824) at org.influxdb.impl.InfluxDBImpl.query(InfluxDBImpl.java:559) at com.github.hgomez.influxdb.InfluxDBFetcher.main(InfluxDBFetcher.java:64) Caused by: java.net.SocketTimeoutException: timeout at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:143) at okio.AsyncTimeout.access$newTimeoutException(AsyncTimeout.kt:162) at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:335) at okio.RealBufferedSource.indexOf(RealBufferedSource.kt:427) at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.kt:320) at okhttp3.internal.http1.HeadersReader.readLine(HeadersReader.kt:29) at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:178) at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:106) at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:79) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at org.influxdb.impl.BasicAuthInterceptor.intercept(BasicAuthInterceptor.java:22) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at org.influxdb.impl.GzipRequestInterceptor.intercept(GzipRequestInterceptor.java:48) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.kt:154) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201) at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154) at retrofit2.OkHttpCall.execute(OkHttpCall.java:204) at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:829) ... 3 more Caused by: java.net.SocketTimeoutException: Read timed out at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:283) at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:309) at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350) at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803) at java.base/java.net.Socket$SocketInputStream.read(Socket.java:982) at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:469) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:463) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1421) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1033) at okio.InputStreamSource.read(JvmOkio.kt:90) at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:129) ... 28 more ```

That is actually the same thing like what @hemang-gandhi reported at https://github.com/influxdata/influxdb-java/issues/544. This patch applies the solution provided by @asashour to the code base (thank you!).

While we might think about making those timeout values configurable in a future iteration, I believe the values are reasonable defaults and will be good to go.

With kind regards, Andreas.

hgomez commented 3 years ago

So we could use OkHttpClient now. Interesting ;)

Thanks Andreas