openhab / openhab-distro

The binary distribution of openHAB
https://www.openhab.org/
Eclipse Public License 2.0
1.3k stars 394 forks source link

I/O error while writing a response message entity to the container output stream #587

Closed lolodomo closed 3 years ago

lolodomo commented 6 years ago

I installed the snapshot 1099 and everything looks good. The only strange thing I noticed is this regular exception:

2017-11-26 11:10:57.155 [ERROR] [ersey.server.ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: org.eclipse.jetty.io.EofException
        at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) [15:com.eclipsesource.jaxrs.publisher:5.3.1.201602281253]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [85:org.eclipse.jetty.security:9.3.22.v20171030]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.Server.handle(Server.java:534) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at java.lang.Thread.run(Thread.java:745) [?:?]
Caused by: org.eclipse.jetty.io.EofException
        at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:375) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.SelectChannelEndPoint$3.run(SelectChannelEndPoint.java:107) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(SelectChannelEndPoint.java:193) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processSelected(ManagedSelector.java:283) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) ~[90:org.eclipse.jetty.util:9.3.22.v20171030]
        ... 5 more
Caused by: java.io.IOException: Relais brisé (pipe)
        at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:?]
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:?]
        at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:?]
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) ~[?:?]
        at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:375) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.SelectChannelEndPoint$3.run(SelectChannelEndPoint.java:107) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(SelectChannelEndPoint.java:193) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processSelected(ManagedSelector.java:283) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) ~[90:org.eclipse.jetty.util:9.3.22.v20171030]
        ... 5 more

Or this one:

2017-11-26 11:17:39.870 [ERROR] [ersey.server.ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: java.nio.channels.ClosedChannelException
        at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [177:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [178:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) [15:com.eclipsesource.jaxrs.publisher:5.3.1.201602281253]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [85:org.eclipse.jetty.security:9.3.22.v20171030]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.Server.handle(Server.java:534) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [90:org.eclipse.jetty.util:9.3.22.v20171030]
        at java.lang.Thread.run(Thread.java:745) [?:?]
Caused by: java.nio.channels.ClosedChannelException
        at org.eclipse.jetty.util.IteratingCallback.close(IteratingCallback.java:427) ~[90:org.eclipse.jetty.util:9.3.22.v20171030]
        at org.eclipse.jetty.server.HttpConnection.onClose(HttpConnection.java:489) ~[87:org.eclipse.jetty.server:9.3.22.v20171030]
        at org.eclipse.jetty.io.ssl.SslConnection.onClose(SslConnection.java:217) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.SelectorManager.connectionClosed(SelectorManager.java:345) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:785) ~[79:org.eclipse.jetty.io:9.3.22.v20171030]
        ... 6 more

I don't succeeed in identifying my action triggering this error. Maybe an exception that is not caught ? But that seems inside Jetty.

lolodomo commented 6 years ago

Starting several times the snapshot 1104, I was not able to reproduce this issue.

lolodomo commented 6 years ago

I talked too fast, the errors are still there.

lolodomo commented 6 years ago

It is still happening in the snapshot 1119. Any chance to identify what is causing this broken pipe ?

kaikreuzer commented 6 years ago

I doubt that it is related to openHAB. If I understand the log correctly, Jetty is processing an incoming HTTP request and fails to respond because the caller prematurely closed the connection. You should check what devices you have in your network that do those requests. Most likely these are UPnP devices as they are "asked" to start communicating with openHAB.

The easiest "fix" on openHAB side is probably that we reconfigure the logger to NOT log such stuff at all.

lolodomo commented 6 years ago

The only new UPnP device I installed recently is the Google Home mini.

lolodomo commented 6 years ago

@kaikreuzer : any chance that it could be OH cloud ?

Seb-Maier commented 6 years ago

I also get this error after installing "apt-get install libpcap-dev" for the amazon dash button. Maybe it's a hint...

kaikreuzer commented 6 years ago

any chance that it could be OH cloud ?

I doubt it, at least not "generic" requests from it, as I use it myself and don't have such errors in the log...

cweitkamp commented 6 years ago

Today I got one of these too. It was not the first time. I navigated through Paper UI a lot while the exception occurred.

2018-01-18 21:56:31.490 [ERROR] [ersey.server.ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: org.eclipse.jetty.io.EofException
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) [182:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) [182:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) [182:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) [182:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) [182:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [181:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [182:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [182:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [179:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [179:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [179:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [179:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [179:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) [19:com.eclipsesource.jaxrs.publisher:5.3.1.201602281253]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) [92:org.eclipse.jetty.servlet:9.3.22.v20171030]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) [92:org.eclipse.jetty.servlet:9.3.22.v20171030]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) [195:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [89:org.eclipse.jetty.security:9.3.22.v20171030]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) [195:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) [92:org.eclipse.jetty.servlet:9.3.22.v20171030]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) [195:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.Server.handle(Server.java:534) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [94:org.eclipse.jetty.util:9.3.22.v20171030]
    at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:147) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:731) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) ~[94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) ~[94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:519) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:745) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:801) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:235) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:219) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:325) ~[?:?]
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) ~[?:?]
    at org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:116) ~[?:?]
    at org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:79) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:105) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:60) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[?:?]
    ... 46 more
Caused by: java.io.IOException: Datenübergabe unterbrochen (broken pipe)
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:?]
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) ~[?:?]
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:147) ~[83:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:731) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) ~[94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) ~[94:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:519) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:745) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:801) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:235) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:219) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) ~[91:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:325) ~[?:?]
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) ~[?:?]
    at org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:116) ~[?:?]
    at org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:79) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:105) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:60) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[?:?]
    ... 46 more
george-harness commented 6 years ago

did someone find the source of the problem? I getting those too.

d0x7 commented 6 years ago

Same over here, but only for the second (and ongoing requests). The first request works just fine, after that jersey starts printing out those exceptions and not handling any requests anymore.

zaier75 commented 6 years ago

any response

olemr commented 6 years ago

These exceptions only happen to me if any browsers are connected to port 8080.

gelsas commented 6 years ago

Got the same issue but i can not figure out what is causing it

kaikreuzer commented 6 years ago

Besides seeing those messages in the log: Does anybody experience any functional issue from this? If not, I would tend to simply reconfigure the logger to simply swallow those messages...

zaier75 commented 6 years ago

Hi, For me, the first request work and the second no, so I found the cause, the person who made the code has declared the response as a static field of the web service class, so the first request work and the second error beacause the output stream is closed.

sipvoip commented 6 years ago

It is not just a log issue, when I get the error I also lose the connnection.

kaikreuzer commented 6 years ago

@sipvoip Which connection exactly are you loosing? From which client and what's the effect of it?

sipvoip commented 6 years ago

I have been losing connecton with my iOS clients, when I look at the logs I see:

018-03-26 12:14:14.062 [WARN ] [org.eclipse.jetty.server.HttpChannel] - //10.88.64.4:8080/rest/sitemaps/christian_tabler/christian_tabler
javax.servlet.ServletException: javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:88) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.Server.handle(Server.java:534) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[?:?]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[?:?]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
    ... 13 more
Caused by: org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:106) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
    at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[?:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[?:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[?:?]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[?:?]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
    ... 13 more
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down
    at org.jvnet.hk2.internal.ServiceLocatorImpl.checkState(ServiceLocatorImpl.java:2288) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandleImpl(ServiceLocatorImpl.java:629) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:622) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:640) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:103) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
    at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[?:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[?:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[?:?]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[?:?]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
    ... 13 more
2018-03-26 12:14:15.835 [WARN ] [eclipse.jetty.servlet.ServletHandler] - 
javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) [15:com.eclipsesource.jaxrs.publisher:5.3.1.201602281253]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [85:org.eclipse.jetty.security:9.3.22.v20171030]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) [88:org.eclipse.jetty.servlet:9.3.22.v20171030]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.Server.handle(Server.java:534) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:106) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
    at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[177:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[177:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[177:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[178:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[178:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    ... 31 more
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down
    at org.jvnet.hk2.internal.ServiceLocatorImpl.checkState(ServiceLocatorImpl.java:2288) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandleImpl(ServiceLocatorImpl.java:629) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:622) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:640) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:103) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
    at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[177:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[177:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[177:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[178:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[178:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[175:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    ... 31 more
2018-03-26 12:14:15.854 [WARN ] [org.eclipse.jetty.server.HttpChannel] - //10.88.64.4:8080/rest/sitemaps/christian_tabler/0001
javax.servlet.ServletException: javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:88) [191:org.ops4j.pax.web.pax-web-jetty:6.0.7]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.Server.handle(Server.java:534) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [87:org.eclipse.jetty.server:9.3.22.v20171030]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [79:org.eclipse.jetty.io:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [90:org.eclipse.jetty.util:9.3.22.v20171030]
    at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[?:?]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[?:?]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
    ... 13 more
Caused by: org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down

    at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:106) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
    at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[?:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[?:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[?:?]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[?:?]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
    ... 13 more
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1146,1147,761547999) has been shut down
    at org.jvnet.hk2.internal.ServiceLocatorImpl.checkState(ServiceLocatorImpl.java:2288) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandleImpl(ServiceLocatorImpl.java:629) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:622) ~[?:?]
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:640) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:103) ~[?:?]
    at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
    at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[?:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[?:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[?:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[?:?]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[?:?]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[?:?]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[?:?]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) ~[?:?]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) ~[?:?]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) ~[?:?]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) ~[?:?]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[?:?]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
    ... 13 more
sipvoip commented 6 years ago

I have 84,889 of them in less than 24 hours.

[root@lisa openhab2]# cat openhab.log |grep "has been shut down" |wc 84889 618477 9495765

Note, I think I see it so much in my environment because I have 22 tablets running openHAB Android app.

kaikreuzer commented 6 years ago

(@sipvoip Please use triple code fences (```)when posting logs - that makes it much better readable.)

Just to understand your situation: You have 22 Android apps, which are running fine (but which might cause those log entries) and an iOS app that disconnects (what does that mean? Do you have to restart it?).

sipvoip commented 6 years ago

Sorry about that.

On the iOS app, I see in red "Error Request failed: internal server error (500)" I don't have to restart the client, I just wait for a second and hit something else and it works again. There are no 500 errors in the logs, the only thing I see is the errors I posted with "has been shut down".

HolgerHees commented 6 years ago

Currently I’m running openhab 2.1 without any problems.

But if I try to upgrade to openhab 2.2 or openhab 2.3 snapshot (1246) I get a lot of

2018-04-01 18:20:35.298 [WARN ] [org.eclipse.jetty.server.HttpChannel] - //mydomain.de/rest/sitemaps/_default/0201
javax.servlet.ServletException: javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_25,26,610229533) has been shut down

it is mostly if I use Habdroid. A lot of rest requests ( 1 of 10) are answered with a 500 or 503. In most of the cases Habdroid is requesting the same resource again so it works at the end. And it is not limited to the rest url in the exception. In most cases a request to my sitemap is failing but I think the reason is because the sitemap is the most request url.

exception.log exception_debug.log

DonStefano commented 6 years ago

I also have lots of those errors in my log. After about 2 days the android app and the basic UI become unresponsible because those exceptions have eaten up my memory (2GB EXC VM running Ubuntu 16.04).

omatzyo commented 6 years ago

I'm getting the same error as that which started this thread. In my case it occurs when I use the alexa skill to run device discovery. It is connected via the myopenhab.org cloud. It then fails to find any devices. This has been working previously.

I am running 2.3.0-SNAPSHOT Build #1281

The error repeats several times and only/instantly when I run discovery.

2018-05-19 17:18:32.432 [ERROR] [ersey.server.ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: org.eclipse.jetty.io.EofException
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) [170:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) [170:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) [170:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) [170:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) [170:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [169:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [170:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [170:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [167:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [167:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [167:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [167:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [167:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) [15:com.eclipsesource.jaxrs.publisher:5.3.1.201602281253]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) [85:org.eclipse.jetty.servlet:9.3.21.v20170918]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) [85:org.eclipse.jetty.servlet:9.3.21.v20170918]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) [183:org.ops4j.pax.web.pax-web-jetty:6.0.9]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [82:org.eclipse.jetty.security:9.3.21.v20170918]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) [183:org.ops4j.pax.web.pax-web-jetty:6.0.9]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) [85:org.eclipse.jetty.servlet:9.3.21.v20170918]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) [183:org.ops4j.pax.web.pax-web-jetty:6.0.9]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.Server.handle(Server.java:534) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [88:org.eclipse.jetty.util:9.3.21.v20170918]
    at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:147) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:739) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) ~[88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) ~[88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:519) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:750) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:801) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:235) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:219) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:325) ~[?:?]
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) ~[?:?]
    at org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:116) ~[?:?]
    at org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:79) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:105) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:60) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[?:?]
    ... 46 more
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:148) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) ~[?:?]
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:179) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:147) ~[77:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:739) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) ~[88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) ~[88:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:519) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:750) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:801) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:235) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:219) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) ~[84:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:325) ~[?:?]
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) ~[?:?]
    at org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:116) ~[?:?]
    at org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:79) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:105) ~[?:?]
    at org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:60) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[?:?]
    ... 46 more
omatzyo commented 6 years ago

After some more debugging this error is related to resources. I'm running on a raspi3. During device discovery the CPU jumps up to >300%. After scaling back a little the task completed successfully and without the error.

spy0r commented 6 years ago

i get those errors (ServiceLocator has been shut down) as well in the 2.3 stable version. At the same time i get the "Offline: waiting for connection to become available" message in my browser, looking at the basic UI.

is there any way i can dig deeper to find out if it is caused by my sitemap, a special item or maybe a rest-api call? I read many threads about that errors and i tried to get rid of them (without setting the log level to error in org.eclipse.jetty), but was not successful.

thanks!

sagittarius0815 commented 6 years ago

@omatzyo

After scaling back a little the task completed successfully and without the error.

I also observed high cpu and memory usage for task java on my raspi3. What have you scaled back? Can you explain an little bit?

Skinah commented 6 years ago

I started getting these as well just after I installed the Kodi and Logitech Harmony Hub bindings. I suspect it is the latter of the two but still doing testing to narrow down the cause. I am running Openhab 2.3 Stable on a pi2 with openhabian.

nesys commented 5 years ago

same issue here. I used to browse via iOS app, so maybe this is related. But not sure and I don't have any correlation to share. Here my logs:

2018-08-24 19:27:38.508 [ERROR] [ersey.server.ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: org.eclipse.jetty.io.EofException
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) [163:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) [163:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) [163:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) [163:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) [163:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [162:org.glassfish.jersey.core.jersey-common:2.22.2]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [163:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [163:org.glassfish.jersey.core.jersey-server:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [160:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [160:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [160:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [160:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [160:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
    at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) [15:com.eclipsesource.jaxrs.publisher:5.3.1.201602281253]
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) [78:org.eclipse.jetty.servlet:9.3.21.v20170918]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584) [78:org.eclipse.jetty.servlet:9.3.21.v20170918]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) [175:org.ops4j.pax.web.pax-web-jetty:6.0.9]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [75:org.eclipse.jetty.security:9.3.21.v20170918]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284) [175:org.ops4j.pax.web.pax-web-jetty:6.0.9]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) [78:org.eclipse.jetty.servlet:9.3.21.v20170918]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) [175:org.ops4j.pax.web.pax-web-jetty:6.0.9]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.Server.handle(Server.java:534) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [80:org.eclipse.jetty.util:9.3.21.v20170918]
    at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:147) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:731) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) ~[80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) ~[80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:519) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:745) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:801) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:235) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:219) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:496) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at java.io.OutputStream.write(OutputStream.java:75) ~[?:?]
    at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:320) ~[?:?]
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:218) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:294) ~[?:?]
    at org.eclipse.smarthome.io.rest.core.internal.GsonProvider.writeTo(GsonProvider.java:71) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[?:?]
    ... 46 more
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:148) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) ~[?:?]
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:179) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:147) ~[69:org.eclipse.jetty.io:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:731) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) ~[80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) ~[80:org.eclipse.jetty.util:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:519) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:745) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:801) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:235) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:219) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:496) ~[77:org.eclipse.jetty.server:9.3.21.v20170918]
    at java.io.OutputStream.write(OutputStream.java:75) ~[?:?]
    at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:320) ~[?:?]
    at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:218) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:294) ~[?:?]
    at org.eclipse.smarthome.io.rest.core.internal.GsonProvider.writeTo(GsonProvider.java:71) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[?:?]
    at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[?:?]
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[?:?]
    ... 46 more
maaf81 commented 5 years ago

I am facing a similar issue when i want to use my 6 dash buttons. In one of the first posts @Seb-Maier already mentioned a similiar problem. When i deactivate the dash button addon everything's fine.

But my openhab 2.3 stable on a RPI3 behaves very differently when the addon is activated. Sometimes everything seems to work fine on startup. Sometimes no KNX link is established (no error in the log related to Calimero or whatever) or the Classic UI or any other UI is unavailable.

In other situations the standard sitemap is not available (also no error in the log related to that service) and the IOS app shows Error 503 - service unavaible. It also seems to have side-effects on the rules engine. I do have several actions triggered by the astro binding. Sometimes they do work, sometime they don't work.

I have no other devices than the dash buttons using a "triggered" action like this in rules. Channel "amazondashbutton:dashbutton:xx-xx-xx-xx-xx-xx:press" triggered Is that way still experimental? I have seen "work arounds" linking the channel to an item and continue with the item state in rules. I wanted to give that i try but i was facing other problems and stopped to go ahead.

Does anyone made similar experiences?

Julian0o commented 5 years ago

I have the same issue. every few days. As sideissue i have the problem that my knx lights won't change.

2018-11-06 21:35:04.778 [ome.event.ItemCommandEvent] - Item 'Licht_WZ_Esstisch' received command ON 2018-11-06 21:35:04.786 [ome.event.ItemCommandEvent] - Item 'Licht_WZ_Esstisch' received command 60 2018-11-06 21:35:04.796 [nt.ItemStatePredictedEvent] - Licht_WZ_Esstisch predicted to become 100

I don't know if the rp3+ has not enough ram because if i restart the openhab service everything is working again.

hamwong commented 5 years ago

I think it may due to you use http browse basicUI and load/connect to https link inside will mostly have this alarm log, when browse basicUI by https, alarm no happen anymore

Julian0o commented 5 years ago

Its also when i send comands via homekit.

KHOne23 commented 5 years ago

I found out that this seems to be because of the MS VSC Tool. Because when i tryed to create a new rule and VSC tryed to get information from Openhab about the Items then the Error in Log are written.

KHOne23 commented 5 years ago

When i use in VSC https and Port 8443 the Error is gone

ghost commented 5 years ago

This issue remains in version 2.4 as well.

2018-12-20 15:53:21.541 [ERROR] [ersey.server.ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream.

I can reproduce the error when closing the openHab iOS app. Seems like OH doesn't think the connection is closed properly. There does not seem to be any loss of functionality.

bofhdk commented 5 years ago

I'm seeing a bunch of these whenever I try to activate a rule:

Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3438,3439,1764817492) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3439,3440,450457073) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3440,3441,222985669) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3441,3442,1604277909) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3442,3443,1344432195) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3443,3444,2027870943) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3444,3445,1099130894) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3446,3447,178995598) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3447,3448,1187850360) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3448,3449,257274732) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3449,3450,76294443) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3450,3451,223869001) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3451,3452,1352419587) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3452,3453,1074198503) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3453,3454,738777901) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3454,3455,994863944) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3456,3457,1524386078) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3457,3458,19395408) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3458,3459,2019223902) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3459,3460,1302481449) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3460,3461,747698019) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3461,3462,434463903) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3462,3463,1337085013) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3463,3464,1738525575) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3464,3465,742478089) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3465,3466,1783035549) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3466,3467,1122230096) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3467,3468,1353969257) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3468,3469,2026326173) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3469,3470,780084947) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3470,3471,215613027) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3471,3472,375436964) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3472,3473,719798830) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3473,3474,1203866551) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3474,3475,1470693757) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3475,3476,970357762) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3476,3477,1160828785) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3477,3478,1503048191) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3478,3479,799843822) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3479,3480,1368597719) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3480,3481,695928759) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3481,3482,1321151270) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3482,3483,583238460) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3484,3485,2089997190) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3485,3486,226003597) has been shut down
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_3486,3487,1195801365) has been shut down

rule:

rule "Good morning all!"
when
        Item Goodmorning changed to ON
then
    // logInfo("goodnight", "It's wakey-wakey-time!")

    // sendCommand(FF_BR_Floalt_ColorTemperature,0)
    // sendCommand(FF_BR_Floalt_Brightness,100)
    // sendCommand(gLt_GF_ST_dm,100)
    // sendCommand(gLt_GF_ST_clr,0)
    // sendCommand(gLt_GF_OF_dm,100)
    // sendCommand(gLt_GF_KI,100)
    // sendCommand(VSX1130_Zone1Power,ON)
    // Thread::sleep(2000)
    // sendCommand(VSX1130_Zone1InputSource,"02")
    // sendCommand(VSX1130_Zone1VolumeDb,"-10")
    // Goodmorning.sendCommand(OFF)
    // sendCommand(Goodmorning,OFF)
end

...I've tried commenting out one line at a time to see if that helped - no joy. Not even the initial logInfo gets executed.

Trigger is:

sitemap:

./sitemaps/My.sitemap:        Switch item=Goodmorning label="Good morning"

items:

./items/My.items:Switch Goodmorning "Wakey-wakey" <bedroom_orange>

openhab version:

# apt show openhab2|head -2
Package: openhab2
Version: 2.4.0-1

lsb-release

# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
wwebers commented 5 years ago

Same here. Made a fresh and clean install of 2.4 and get a lot of this. Don't think it's OpenHAB related, but jetty (resp. the hole HK2 bundle). Couldn't analyze the full thing, but looks like Jetty is trying to release a handler which has already the "SHUTDOWN" state. As this one appears constantly when I startup OpenHAB, I would guess it's a kind of a race condition inside Jetty. Do we have to use Jetty in the first place?

J-u-n-o commented 5 years ago

Count me in... Already cleared cache, readded my zwave dongle and its things (did not repair) For Jython I am at unstable. Revert to 2.4 stable?

Package: openhab2 Version: 2.5.0~S1479-1

openjdk version "1.8.0_152" OpenJDK Runtime Environment (Zulu Embedded 8.25.0.76-linux-aarch32hf) (build 1.8.0_152-b76) OpenJDK Client VM (Zulu Embedded 8.25.0.76-linux-aarch32hf) (build 25.152-b76, mixed mode, Evaluation)

PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"

Try to get Jython running...

2018-12-25 23:01:18.051 [WARN ] [org.eclipse.jetty.server.HttpChannel] - /rest/sitemaps/me/me
javax.servlet.ServletException: javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_35,36,1411840) has been shut down

        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:88) ~[?:?]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.Server.handle(Server.java:531) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) [84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) [84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) [75:org.eclipse.jetty.io:9.4.11.v20180605]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) [75:org.eclipse.jetty.io:9.4.11.v20180605]
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) [75:org.eclipse.jetty.io:9.4.11.v20180605]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) [87:org.eclipse.jetty.util:9.4.11.v20180605]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) [87:org.eclipse.jetty.util:9.4.11.v20180605]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) [87:org.eclipse.jetty.util:9.4.11.v20180605]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) [87:org.eclipse.jetty.util:9.4.11.v20180605]
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) [87:org.eclipse.jetty.util:9.4.11.v20180605]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) [87:org.eclipse.jetty.util:9.4.11.v20180605]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) [87:org.eclipse.jetty.util:9.4.11.v20180605]
        at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: javax.servlet.ServletException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_35,36,1411840) has been shut down

        at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[82:org.eclipse.jetty.security:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:293) ~[?:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
        ... 15 more
Caused by: org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions.  They are:
1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_35,36,1411840) has been shut down

        at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:106) ~[?:?]
        at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
        at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
        at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[171:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[171:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[171:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[172:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[172:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[82:org.eclipse.jetty.security:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:293) ~[?:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
        ... 15 more
Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_35,36,1411840) has been shut down
        at org.jvnet.hk2.internal.ServiceLocatorImpl.checkState(ServiceLocatorImpl.java:2288) ~[?:?]
        at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandleImpl(ServiceLocatorImpl.java:629) ~[?:?]
        at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:622) ~[?:?]
        at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:640) ~[?:?]
        at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:103) ~[?:?]
        at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173) ~[?:?]
        at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526) ~[?:?]
        at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532) ~[171:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549) ~[171:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319) ~[171:org.glassfish.jersey.core.jersey-common:2.22.2]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[172:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[172:org.glassfish.jersey.core.jersey-server:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[169:org.glassfish.jersey.containers.jersey-container-servlet-core:2.22.2]
        at com.eclipsesource.jaxrs.publisher.internal.ServletContainerBridge.service(ServletContainerBridge.java:76) ~[?:?]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[?:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[82:org.eclipse.jetty.security:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:293) ~[?:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) ~[85:org.eclipse.jetty.servlet:9.4.11.v20180605]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) ~[84:org.eclipse.jetty.server:9.4.11.v20180605]
        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[?:?]
        ... 15 more
JueBag commented 5 years ago

I'm on the OH 2.4 stable and do get those "java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated..." messages as well. Makes the system unresponsive! Restarting didn't help. IMHO it can't be VSC since it occurs here even if VSC is not used (PC is off).

wwebers commented 5 years ago

@JueBag Agree, does nothing have to do with VSC. I do not even have any SAMBA share. BTW: I can see that @Juno72 is using the Zulu OpenJDK version. I'm using the Oracle version. I guess we can ignore the JVM framework as well. As I mentioned, I'm pretty sure it's Jetty related. Or at least how Jetty is used within OpenHAB.

Gugiman commented 5 years ago

Removing (or renaming) /var/lib/openhab2/config/org/openhab/addons.config, restart openhab and let openhab configure it new solved the problem (atm). So maybe it's a problem caused by obsolete configurations. Caution: In my case, every binding had to be installed again but their configuration and also the already configured things are still there and working. Consider making a backup before with openhab-cli. So take some time, delete or rename this file and instal all bindings again. (addons.config lists them so you don't have to remember them)

https://community.openhab.org/t/multiexception-error-in-latest-snapshot/37758/7

wborn commented 5 years ago

If you see the ServletException: javax.servlet.ServletException: A MultiException has 1 exceptions it may be caused by load resulting from openHAB continuously trying to install addons that don't exist.

With 2.5.0-SNAPSHOT it should log an error for that (https://github.com/openhab/openhab-core/pull/459). See also this community topic.

fkemps commented 5 years ago

I'm experiencing the same errors on 2.4-stable when I close/start or refresh OpenHab client on IOS.

A network capture shows that OpenHab client sends RST packets and doesn't teardown the HTTP connection gracefully.

It happens after the /rest/sitemaps/default/default request, which receives a partial answer. Think due to the RST that's send half way.

Requests in order on the same TCP stream: GET / HTTP/1.1 GET /start/index HTTP/1.1 GET /rest/bindings HTTP/1.1 GET /rest/sitemaps HTTP/1.1 GET /rest/sitemaps/default/default HTTP/1.1

The next TCP session requests the same page which is fully send. `GET /rest/sitemaps/default/default HTTP/1.1` Not sure why OpenHab client sends the RST
tnemrap commented 5 years ago

Removing (or renaming) /var/lib/openhab2/config/org/openhab/addons.config, restart openhab and let openhab configure it new solved the problem (atm). So maybe it's a problem caused by obsolete configurations. Caution: In my case, every binding had to be installed again but their configuration and also the already configured things are still there and working. Consider making a backup before with openhab-cli. So take some time, delete or rename this file and instal all bindings again. (addons.config lists them so you don't have to remember them)

https://community.openhab.org/t/multiexception-error-in-latest-snapshot/37758/7

Can someone confirm that this solves the issue.

nesys commented 5 years ago

OH2 #1502 here, and still the same error. It seems related to the IOS app somehow

phuongpham commented 5 years ago

I also having this issue

nesys commented 5 years ago

Removing (or renaming) /var/lib/openhab2/config/org/openhab/addons.config, restart openhab and let openhab configure it new solved the problem (atm). So maybe it's a problem caused by obsolete configurations. Caution: In my case, every binding had to be installed again but their configuration and also the already configured things are still there and working. Consider making a backup before with openhab-cli. So take some time, delete or rename this file and instal all bindings again. (addons.config lists them so you don't have to remember them) https://community.openhab.org/t/multiexception-error-in-latest-snapshot/37758/7

Can someone confirm that this solves the issue.

this is NOT solving the issue. At least for me

nesys commented 5 years ago

Please check the configuration of you App. Remote URL should behttps://myopenhab.org the Local URL should be http://192.168.x.x:8080 and see if this solves the issue.