search

magro / memcached-session-manager

A tomcat session manager that backups sessions in memcached and pulls them from there if asked for unknown sessions
Apache License 2.0
760 stars 346 forks source link

session manager constantly failed when Tomcat class auto-reload is ON #364

Closed balmydrizzle closed 6 years ago

balmydrizzle commented 6 years ago

We use httpd+Tomcat (connected by using modjk connector) both on our product and lab environments. The only difference is class-auto-reload is off on product side whereas it's on on lab side. In our lab side, memcached session manager constantly failed and we have to restart httpd+Tomcat to recover it. But on product side, it almost never failed. When failed, the most frequent error sentence is: "Illegal access: this web application instance has been stopped already". Below is one of the sample error messages dumped in catalina.out ( a bit lengthy):

INFO: Illegal access: this web application instance has been stopped already. Could not load org.jasig.cas.client.validation.AssertionImpl. The eventual fo java.lang.IllegalStateException at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1566) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:447) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:561) at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129) at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61) at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:589) at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:213) at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:261) at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:169) at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:126) at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1136) at de.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:609) at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:216) at org.apache.catalina.connector.Request.doGetSession(Request.java:2419) at org.apache.catalina.connector.Request.getSession(Request.java:2157) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:833) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:844) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:224) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:224) at newdata.Index.doGet(Index.java:104) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at newdata.CharsetFilter.doFilter(CharsetFilter.java:27) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at newdata.CharsetFilter.doFilter(CharsetFilter.java:27) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:170) at com.googlecode.psiprobe.Tomcat60AgentValve.invoke(Tomcat60AgentValve.java:30) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:311) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:776) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:705) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:898) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690) at java.lang.Thread.run(Thread.java:662)

magro commented 6 years ago

The question is why the application had been stopped, right?

magro commented 6 years ago

Apart from this, is this still relevant, or can we close this issue?

balmydrizzle commented 6 years ago

Yes. I'll close it. We've found the most possib;e cause of this problem could be due to that server has two ips with different C-class subnet and contantly makes memcached request lost. We delete one of the ips, and it seems to be "better" but not totally "well", though. Anyway, we could live this current situation. Thanks for reply, anyway.

magro commented 6 years ago

Thanks for sharing your insights!