search

ari-ban / issue-test

0 stars 0 forks source link

Internal error causes massive java.io.EOFException: null #1952

Open arinban opened 7 years ago

arinban commented 7 years ago

Hi ! I randomly got massive exception logs on our servers which completly break down them. I can't figure out what causes this and we hope someone can help us. If you need more informations on our setup, please tell me. I don't want to disturb you with any fake clue. The only thing "special" I could found is that we are actually using ssl. Thanks in advance and have a nice day.

Internal error for https://myurl java.io.EOFException: null at org.glassfish.grizzly.nio.transport.TCPNIOTransport.read(TCPNIOTransport.java:617) at org.glassfish.grizzly.nio.transport.TCPNIOTemporarySelectorReader.readNow0(TCPNIOTemporarySelectorReader.java:65) at org.glassfish.grizzly.nio.tmpselectors.TemporarySelectorReader.read0(TemporarySelectorReader.java:171) at org.glassfish.grizzly.nio.tmpselectors.TemporarySelectorReader.read0(TemporarySelectorReader.java:141) at org.glassfish.grizzly.nio.tmpselectors.TemporarySelectorReader.read(TemporarySelectorReader.java:113) at org.glassfish.grizzly.nio.tmpselectors.TemporarySelectorReader.read(TemporarySelectorReader.java:75) at org.glassfish.grizzly.AbstractReader.read(AbstractReader.java:72) at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleRead(TCPNIOTransportFilter.java:77) at org.glassfish.grizzly.filterchain.TransportFilter.handleRead(TransportFilter.java:173) at org.glassfish.grizzly.ssl.SSLBaseFilter$SSLTransportFilterWrapper.handleRead(SSLBaseFilter.java:1132) at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119) at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284) at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201) at org.glassfish.grizzly.filterchain.DefaultFilterChain.read(DefaultFilterChain.java:352) at org.glassfish.grizzly.filterchain.FilterChainContext.read(FilterChainContext.java:736) at org.glassfish.grizzly.http.io.InputBuffer.blockingRead(InputBuffer.java:1103) at org.glassfish.grizzly.http.server.io.ServerInputBuffer.blockingRead(ServerInputBuffer.java:95) at org.glassfish.grizzly.http.io.InputBuffer.fill(InputBuffer.java:1127) at org.glassfish.grizzly.http.io.InputBuffer.read(InputBuffer.java:348) at org.glassfish.grizzly.http.server.NIOInputStreamImpl.read(NIOInputStreamImpl.java:83) at org.glassfish.jersey.message.internal.EntityInputStream.read(EntityInputStream.java:102) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream.read(ReaderInterceptorExecutor.java:296) at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.ensureLoaded(ByteSourceJsonBootstrapper.java:522) at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.detectEncoding(ByteSourceJsonBootstrapper.java:129) at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.constructParser(ByteSourceJsonBootstrapper.java:246) at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java:1271) at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:810) at com.fasterxml.jackson.jaxrs.base.ProviderBase._createParser(ProviderBase.java:822) at com.fasterxml.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:763) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.invokeReadFrom(ReaderInterceptorExecutor.java:256) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.aroundReadFrom(ReaderInterceptorExecutor.java:235) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:155) at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundReadFrom(MappableExceptionWrapperInterceptor.java:74) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:155) at org.glassfish.jersey.message.internal.MessageBodyFactory.readFrom(MessageBodyFactory.java:1085) at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:874) at org.glassfish.jersey.server.ContainerRequest.readEntity(ContainerRequest.java:271) at org.glassfish.jersey.server.internal.inject.EntityParamValueFactoryProvider$EntityValueFactory.provide(EntityParamValueFactoryProvider.java:96) at org.glassfish.jersey.server.spi.internal.ParamValueFactoryWithSource.provide(ParamValueFactoryWithSource.java:71) at org.glassfish.jersey.server.spi.internal.ParameterValueHelper.getParameterValues(ParameterValueHelper.java:94) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$AbstractMethodParamInvoker.getParamValues(JavaResourceMethodDispatcherProvider.java:127) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:593) at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:573) at java.lang.Thread.run(Thread.java:745)

arinban commented 6 years ago
arinban commented 7 years ago

@rlubke Commented Looks like a -1 was returned from a read call. Possible client disconnect. If you can reproduce the issue regularly, please provide a test case, otherwise, I'm not sure there is a problem here.

arinban commented 7 years ago

@Talentyre Commented Hi Ryan, thank you for your answer. It is difficult for us to reproduce this issue as it only happens on production server with hundred connections... There is really a problem here because this issue broke down our servers. We actually found a workaround : Changing "SameThreadIOStrategy" to "WorkerThreadIOStrategy" in this call : transport.setIOStrategy(WorkerThreadIOStrategy.getInstance()); Now, we still have the exception log, but this don't lead to a crash anymore. We suppose there is a catch issue or something when we use SameThreadIOStrategy. Thanks for your concern, have a nice day.