search

CUTR-at-USF / onebusaway-application-modules

OneBusAway, modified for a Tampa, Fl deployment
https://github.com/OneBusAway/onebusaway-application-modules/wiki
Other
2 stars 1 forks source link

OneBusAway completely crashed on 6/28 - possible memory leak #28

Closed barbeau closed 11 years ago

barbeau commented 11 years ago

OneBusAway Tomcat completely imploded, returning 404 for OBA website as well as API (i.e., apps couldn't access it)

There were a number of org.onebusaway.exceptions.NoSuchTripServiceExceptions:

Jun 28, 2013 10:04:10 AM com.caucho.hessian.server.HessianSkeleton invoke
WARNING: org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_830853
org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_830853
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.createArrivalAndDepartureQuery(TransitDataServiceImpl.java:666)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.getArrivalAndDepartureForStop(TransitDataServiceImpl.java:294)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceExceptionInterceptor.doBasicProfiling(TransitDataServiceExceptionInterceptor.java:37)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109)
    at org.springframework.remoting.caucho.HessianExporter.doInvoke(HessianExporter.java:198)
    at org.springframework.remoting.caucho.HessianExporter.invoke(HessianExporter.java:118)
    at org.springframework.remoting.caucho.HessianServiceExporter.handleRequest(HessianServiceExporter.java:66)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:774)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
    at java.lang.Thread.run(Unknown Source)
Jun 28, 2013 10:14:26 AM com.caucho.hessian.server.HessianSkeleton invoke
WARNING: org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828346
org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828346
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.createArrivalAndDepartureQuery(TransitDataServiceImpl.java:666)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.getArrivalAndDepartureForStop(TransitDataServiceImpl.java:294)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceExceptionInterceptor.doBasicProfiling(TransitDataServiceExceptionInterceptor.java:37)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109)
    at org.springframework.remoting.caucho.HessianExporter.doInvoke(HessianExporter.java:198)
    at org.springframework.remoting.caucho.HessianExporter.invoke(HessianExporter.java:118)
    at org.springframework.remoting.caucho.HessianServiceExporter.handleRequest(HessianServiceExporter.java:66)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:774)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
    at java.lang.Thread.run(Unknown Source)

Then, this error:


    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
    at java.lang.Thread.run(Unknown Source)
Jun 28, 2013 11:28:52 AM com.caucho.hessian.server.HessianSkeleton invoke
WARNING: org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828383
org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828383
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.createArrivalAndDepartureQuery(TransitDataServiceImpl.java:666)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.getArrivalAndDepartureForStop(TransitDataServiceImpl.java:294)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceExceptionInterceptor.doBasicProfiling(TransitDataServiceExceptionInterceptor.java:37)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109)
    at org.springframework.remoting.caucho.HessianExporter.doInvoke(HessianExporter.java:198)
    at org.springframework.remoting.caucho.HessianExporter.invoke(HessianExporter.java:118)
    at org.springframework.remoting.caucho.HessianServiceExporter.handleRequest(HessianServiceExporter.java:66)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:774)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
    at java.lang.Thread.run(Unknown Source)
Jun 28, 2013 11:29:01 AM com.caucho.hessian.server.HessianSkeleton invoke
WARNING: org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828374
org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828374
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.createArrivalAndDepartureQuery(TransitDataServiceImpl.java:666)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.getArrivalAndDepartureForStop(TransitDataServiceImpl.java:294)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceExceptionInterceptor.doBasicProfiling(TransitDataServiceExceptionInterceptor.java:37)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109)
    at org.springframework.remoting.caucho.HessianExporter.doInvoke(HessianExporter.java:198)
    at org.springframework.remoting.caucho.HessianExporter.invoke(HessianExporter.java:118)
    at org.springframework.remoting.caucho.HessianServiceExporter.handleRequest(HessianServiceExporter.java:66)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:774)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
    at java.lang.Thread.run(Unknown Source)
Jun 28, 2013 11:59:35 AM com.caucho.hessian.server.HessianSkeleton invoke
WARNING: org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828348
org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828348
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.createArrivalAndDepartureQuery(TransitDataServiceImpl.java:666)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.getArrivalAndDepartureForStop(TransitDataServiceImpl.java:294)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceExceptionInterceptor.doBasicProfiling(TransitDataServiceExceptionInterceptor.java:37)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109)
    at org.springframework.remoting.caucho.HessianExporter.doInvoke(HessianExporter.java:198)
    at org.springframework.remoting.caucho.HessianExporter.invoke(HessianExporter.java:118)
    at org.springframework.remoting.caucho.HessianServiceExporter.handleRequest(HessianServiceExporter.java:66)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:774)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
    at java.lang.Thread.run(Unknown Source)
Jun 28, 2013 12:26:32 PM com.caucho.hessian.server.HessianSkeleton invoke
WARNING: org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828380
org.onebusaway.exceptions.NoSuchTripServiceException: No such trip: id=Hillsborough Area Regional Transit_828380
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.createArrivalAndDepartureQuery(TransitDataServiceImpl.java:666)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceImpl.getArrivalAndDepartureForStop(TransitDataServiceImpl.java:294)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
    at org.onebusaway.transit_data_federation.impl.federated.TransitDataServiceExceptionInterceptor.doBasicProfiling(TransitDataServiceExceptionInterceptor.java:37)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy99.getArrivalAndDepartureForStop(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180)
    at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109)
    at org.springframework.remoting.caucho.HessianExporter.doInvoke(HessianExporter.java:198)
    at org.springframework.remoting.caucho.HessianExporter.invoke(HessianExporter.java:118)
    at org.springframework.remoting.caucho.HessianServiceExporter.handleRequest(HessianServiceExporter.java:66)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:774)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
    at java.lang.Thread.run(Unknown Source)
Jun 28, 2013 12:42:26 PM org.apache.tomcat.util.net.AprEndpoint createWorkerThread
INFO: Maximum number of threads (200) created for connector with address null and port 80
Jun 28, 2013 3:50:04 PM org.apache.coyote.http11.Http11AprProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-80
Jun 28, 2013 3:50:04 PM org.apache.coyote.ajp.AjpAprProtocol pause
INFO: Pausing Coyote AJP/1.3 on ajp-8009
Jun 28, 2013 3:50:05 PM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/api] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@3308db1f]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [net.sf.json.AbstractJSON$1] (value [net.sf.json.AbstractJSON$1@a69ea66]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@2f9bd3e1]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [net.sf.json.AbstractJSON$1] (value [net.sf.json.AbstractJSON$1@a69ea66]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@1f737132]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [net.sf.json.AbstractJSON$1] (value [net.sf.json.AbstractJSON$1@a69ea66]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@540c99bf]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [net.sf.json.AbstractJSON$1] (value [net.sf.json.AbstractJSON$1@a69ea66]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@741f5b5]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [net.sf.json.AbstractJSON$1] (value [net.sf.json.AbstractJSON$1@a69ea66]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@7b6a41cb]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [net.sf.json.AbstractJSON$1] (value [net.sf.json.AbstractJSON$1@a69ea66]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@7d05b5b9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [net.sf.json.AbstractJSON$1] (value [net.sf.json.AbstractJSON$1@a69ea66]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/api] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@337e745a]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@1ddc4cf]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jun 28, 2013 3:50:08 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

Then:

SEVERE: The web application [/onebusaway-transit-data-federation-webapp] appears to have started a thread named [Store org.onebusaway.transit_data_federation.services.beans.RouteBeanService.getRouteForId Spool Thread] but has failed to stop it. This is very likely to create a memory leak.
Jun 28, 2013 3:50:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/onebusaway-webapp] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
Jun 28, 2013 3:50:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/onebusaway-webapp] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
Jun 28, 2013 3:50:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/onebusaway-webapp] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
Jun 28, 2013 3:50:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

So it looks like a memory leak, with threads tied up in tasks, and the number of threads exceeding the allowed number.

Full error log is here: https://docs.google.com/file/d/0B8oU647elPShblJxcEZfTHBBUUk/edit?usp=sharing

barbeau commented 11 years ago

No issues yet on HART, so I'm closing this. We can re-open if we encounter this again.