twitter / finatra

Fast, testable, Scala services built on TwitterServer and Finagle
https://twitter.github.io/finatra/
Apache License 2.0
2.27k stars 405 forks source link

Updating finatra showing a lot of io.netty.handler.ssl.SslClosedEngineException: SSLEngine closed already at #580

Closed VAIBHAVCES closed 2 years ago

VAIBHAVCES commented 2 years ago

Recently I was using up an older version of finatra finatra = "20.9.0" but recently after updating it to val finatra = "21.9.0"I am seeing up a lot of below errors , is there any configuration I can follow up to remove this logs OR is there any way to suppress this logs

2021-12-08 14:45:23.858 [finagle/netty4-2-5] DEBUG i.n.h.c.http2.Http2ConnectionHandler - [id: 0x1c4d3866, L:/1.2.3.4 - R:xyz.com/1.2.3.4] Sending GOAWAY failed: lastStreamId '0', errorCode '0', debugData ''. Forcing shutdown of the connection. io.netty.handler.ssl.SslClosedEngineException: SSLEngine closed already at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:861) at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:800) at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:781) at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750) at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742) at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728) at io.netty.handler.codec.http2.Http2ConnectionHandler.close(Http2ConnectionHandler.java:476) at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:622) at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:606) at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:472) at io.netty.channel.DefaultChannelPipeline.close(DefaultChannelPipeline.java:957) at io.netty.channel.AbstractChannel.close(AbstractChannel.java:244) at com.twitter.finagle.netty4.ssl.client.Netty4ClientSslChannelInitializer$$anon$1$$anon$2.run(Netty4ClientSslChannelInitializer.scala:52) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at com.twitter.finagle.util.BlockingTimeTrackingThreadFactory$$anon$1.run(BlockingTimeTrackingThreadFactory.scala:23) Caused by: java.security.cert.CertPathValidatorException: validity check failed at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135) at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:224) at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:144) at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:83) at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292) at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:381) ... 38 common frames omitted Caused by: java.security.cert.CertificateExpiredException: NotAfter: Mon Nov 29 05:33:51 GMT 2021 at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:277) at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:677) at sun.security.provider.certpath.BasicChecker.verifyValidity(BasicChecker.java:190) at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144) at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125) ... 43 common frames omitte

jyanJing commented 2 years ago

Hi @VAIBHAVCES , before I dig into the error, wanted to check with you if you are seeing this issue from the latest version of Finatra? What is the reason you are using an older version of Finatra?

VAIBHAVCES commented 2 years ago

Hi @jyanJing , Yes I started to see this only after using up the new version of finatra 21.9 I have also checked by using older version in that these logs are not visible, this part was updated 6-7months back, at that time I guess this was the latest release that's why currently I am not on latest version, but if latest fix has change for it probably we can update , but in my suggestion as part of backward compatibility this version should also work fine

jyanJing commented 2 years ago

Hi @VAIBHAVCES , which Http version are you using for your client and server? Asking since we removed UseH2 toggles in Finagle 21.8.0, all clients and servers are defaulted to HTTP/2, I was wondering if that is related to the error you are seeing. You could try verifying it by using c.t.finagle.Http.client.withNoHttp2 and c.t.finagle.Http.server.withNoHttp2 respectively to create a client and server.

joybestourous commented 2 years ago

hey @VAIBHAVCES, looks like you haven't replied in a while, is this still an issue you're seeing?

VAIBHAVCES commented 2 years ago

Hi @jyanJing , @joybestourous thanks for your triage , most probably the solution for my problem is the one which @jyanJing suggested to add c.t.finagle.Http.client.withNoHttp2 in my HTTP client , after that at-least I was not able to monitor it in my local setup, I wanted to move my code to production and wanted to monitor it there , but I would prefer closing this issue for now , if I face any problem in future i would prefer to reopen it , thanks again for your efforts and time