The mdba.ala.org.au/index page is accessed once every minute by uptime.ala.org.au to monitor the system.
The rendering of the index page (or another page) may have a memory leak, as consistently fell over for a 4 hour period overnight with the following errors consistently being sent to catalina.out:
2016-10-27 15:58:09,454 DEBUG [AuthenticationCookieUtils] Cookie ALA-Auth not found
2016-10-27 15:58:09,454 DEBUG [UriFilter] No action taken - no matching pattern found for /
2016-10-27 15:58:09,454 DEBUG [UriFilter] Request Uri = '/'
2016-10-27 15:58:09,454 DEBUG [AuthenticationCookieUtils] Cookie ALA-Auth not found
2016-10-27 15:58:09,454 DEBUG [UriFilter] No action taken - no matching pattern found for /
2016-10-27 15:58:09,454 DEBUG [AuthenticationCookieUtils] Cookie ALA-Auth not found
2016-10-27 15:58:09,454 DEBUG [UriFilter] No action taken - no matching pattern found for /
Oct 27, 2016 3:58:19 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [gsp] in context with path [] threw exception
org.codehaus.groovy.grails.web.mapping.exceptions.UrlMappingException: Error mapping onto view [/index]: Error processing GroovyPageView: Error executing tag <sitemesh:captureContent>: Error executing tag <g:img>: PermGen space
at au.org.ala.cas.client.UriFilter.doFilter(UriFilter.java:200)
at au.org.ala.cas.client.UriFilter.doFilter(UriFilter.java:200)
at au.org.ala.cas.client.UriFilter.doFilter(UriFilter.java:200)
at org.jasig.cas.client.session.SingleSignOutFilter.doFilter(SingleSignOutFilter.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.codehaus.groovy.grails.web.pages.exceptions.GroovyPagesException: Error processing GroovyPageView: Error executing tag <sitemesh:captureContent>: Error executing tag <g:img>: PermGen space
... 7 more
Caused by: org.codehaus.groovy.grails.web.taglib.exceptions.GrailsTagException: Error executing tag <sitemesh:captureContent>: Error executing tag <g:img>: PermGen space
at gsp_mdbaHubindex_gsp$_run_closure2.doCall(gsp_mdbaHubindex_gsp.groovy:43)
at gsp_mdbaHubindex_gsp.run(gsp_mdbaHubindex_gsp.groovy:76)
... 7 more
Caused by: org.codehaus.groovy.grails.web.taglib.exceptions.GrailsTagException: Error executing tag <g:img>: PermGen space
at gsp_mdbaHubindex_gsp$_run_closure2$_closure5.doCall(gsp_mdbaHubindex_gsp.groovy:40)
... 9 more
Caused by: java.lang.OutOfMemoryError: PermGen space
There were also OOME instances yesterday while I was testing, but I didn't link them conclusively to the mdba-hub index page.
This is a blocker to migrating other hubs to the same server as the mdba hub as this behaviour is occurring with only two hubs DNS mapped to aws-hubs.ala.org.au (MDBA and OBIS) and it will only get worse if more hubs are mapped to this server.
Restarting the server each day is not working, as the OOME are still occurring within 24 hours after a restart (the last restart was yesterday afternoon when I redeployed OBIS to the server).
The mdba.ala.org.au/index page is accessed once every minute by uptime.ala.org.au to monitor the system.
The rendering of the index page (or another page) may have a memory leak, as consistently fell over for a 4 hour period overnight with the following errors consistently being sent to catalina.out:
There were also OOME instances yesterday while I was testing, but I didn't link them conclusively to the mdba-hub index page.
This is a blocker to migrating other hubs to the same server as the mdba hub as this behaviour is occurring with only two hubs DNS mapped to aws-hubs.ala.org.au (MDBA and OBIS) and it will only get worse if more hubs are mapped to this server.
Restarting the server each day is not working, as the OOME are still occurring within 24 hours after a restart (the last restart was yesterday afternoon when I redeployed OBIS to the server).