search

loki4j / loki-logback-appender

Fast and lightweight implementation of Logback appender for Grafana Loki
https://loki4j.github.io/loki-logback-appender/
BSD 2-Clause "Simplified" License
300 stars 26 forks source link

Loki appender didn't retry on HttpConnectTimeoutException #243

Closed Unknow0 closed 3 months ago

Unknow0 commented 3 months ago

Hi,

we have a connect timeout from time to time at the startup of the application.

shouldn't he have retried in this case ? as it retry in case of ConnectException is seem logical for me to retried here as well, we know for sure the logs wasn't send.

21:01:21,387 |-ERROR in com.github.loki4j.client.pipeline.AsyncBufferPipeline@1624588e - Error while sending Batch #35dbe3f1b75483 (480 bytes) to Loki (https://xxx/loki/api/v1/push) java.net.http.HttpConnectTimeoutException: HTTP connect timed out
    at java.net.http.HttpConnectTimeoutException: HTTP connect timed out
    at  at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:567)
    at  at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:123)
    at  at com.github.loki4j.client.http.JavaHttpClient.send(JavaHttpClient.java:68)
    at  at com.github.loki4j.client.pipeline.AsyncBufferPipeline.sendBatch(AsyncBufferPipeline.java:322)
    at  at com.github.loki4j.client.pipeline.AsyncBufferPipeline.sendStep(AsyncBufferPipeline.java:293)
    at  at com.github.loki4j.client.pipeline.AsyncBufferPipeline.runSendLoop(AsyncBufferPipeline.java:224)
    at  at com.github.loki4j.client.pipeline.AsyncBufferPipeline.lambda$start$3(AsyncBufferPipeline.java:131)
    at  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at  at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.net.http.HttpConnectTimeoutException: HTTP connect timed out
    at  at java.net.http/jdk.internal.net.http.ResponseTimerEvent.handle(ResponseTimerEvent.java:68)
    at  at java.net.http/jdk.internal.net.http.HttpClientImpl.purgeTimeoutsAndReturnNextDeadline(HttpClientImpl.java:1270)
    at  at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:899)
Caused by: java.net.ConnectException: HTTP connect timed out
    at  at java.net.http/jdk.internal.net.http.ResponseTimerEvent.handle(ResponseTimerEvent.java:69)
    at  ... 2 common frames omitted

loki-logback-appender 1.5.1

java -version:

openjdk version "17.0.11" 2024-04-16
OpenJDK Runtime Environment (build 17.0.11+9-alpine-r0)
OpenJDK 64-Bit Server VM (build 17.0.11+9-alpine-r0, mixed mode, sharing)

Regards.

nehaev commented 3 months ago

Hi @Unknow0, thanks for posting this! I was able to reproduce this, will fix shortly.