OutOfMemoryError: PermGen space

[dtrunk90]

In development (local machine) I've created my project and directly after mvn clean tomcat7:run the webapp initializes. Then on first request some work is done by wro4j and I'm getting the out of memory exception.

I'm running Ubuntu 13.04 and my /etc/defaults/tomcat7 contains the following JAVA_OPTS:

-Djava.awt.headless=true -Xmx1024m -XX:+UseConcMarkSweepGC -XX:PermSize=2048m -XX:MaxPermSize=2048m

This is my wro4j config:

preProcessors=less4j,cssUrlRewriting,cssImport,semicolonAppender
postProcessors=jsMin,cssMinJawr
modelFactory=customXml
disableCache=true

With a custom model factory provider registered in META-INF/ro.isdc.model.spi.ModelFactoryProvider:

public class CustomModelFactoryProvider implements ModelFactoryProvider {
    public Map provideModelFactories() {
        final Map map = new HashMap();
        map.put(CustomXmlModelFactory.ALIAS, new CustomXmlModelFactory());
        return map;
    }
}

and a custom xml model factory to load wro.xml directly from classpath:

public class CustomXmlModelFactory extends XmlModelFactory {
    public static final String ALIAS = "customXml";

    @Override
    protected InputStream getModelResourceAsStream() throws IOException {
        final String resourceLocation = "classpath:wro.xml";
        final InputStream stream = new ClasspathUriLocator().locate(resourceLocation);

        if (stream == null) {
            throw new IOException("Invalid resource requested: " + resourceLocation);
        }

        return stream;
    }
}

Configured my webapp without xml (Java Config):

public class WebAppInitializer implements WebApplicationInitializer {
    public @Override void onStartup(ServletContext container) throws ServletException {
        AnnotationConfigWebApplicationContext rootContext = new AnnotationConfigWebApplicationContext();
        rootContext.getEnvironment().setDefaultProfiles("production");
        rootContext.scan("com.example.config");

        container.addListener(new ContextLoaderListener(rootContext));

        ServletRegistration.Dynamic servlet = container.addServlet("DispatcherServlet", DispatcherServlet.class);
        servlet.setInitParameter("contextConfigLocation", "");
        servlet.setLoadOnStartup(1);
        servlet.addMapping("/");

        FilterRegistration charEncodingfilterReg = container.addFilter("CharacterEncodingFilter", CharacterEncodingFilter.class);
        charEncodingfilterReg.addMappingForUrlPatterns(EnumSet.allOf(DispatcherType.class), true, "/*");
        charEncodingfilterReg.setInitParameter("encoding", "UTF-8");
        charEncodingfilterReg.setInitParameter("forceEncoding", "true");

        DelegatingFilterProxy delegatingFilterProxy = new DelegatingFilterProxy("wroFilter");
        delegatingFilterProxy.setTargetFilterLifecycle(true);

        FilterRegistration wroFilterReg = container.addFilter("WebResourceOptimizer", delegatingFilterProxy);
        wroFilterReg.addMappingForUrlPatterns(EnumSet.allOf(DispatcherType.class), true, "/resources/opt/*");
    }
}

I'll set up an example project later and try to reproduce this. But maybe you can take a look at this stacktrace in the meanwhile:

-----------------------------------------
ms     %     Task name
-----------------------------------------
01048  100%  Using ro.isdc.wro.extensions.processor.css.Less4jProcessor@72c8c9ae

Exception in thread "http-bio-8080-exec-3" java.lang.OutOfMemoryError: PermGen space
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
    at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:1189)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1558)
    at com.github.sommeri.less4j.core.parser.ANTLRParser.createLexer(ANTLRParser.java:88)
    at com.github.sommeri.less4j.core.parser.ANTLRParser.parse(ANTLRParser.java:63)
    at com.github.sommeri.less4j.core.parser.ANTLRParser.parseStyleSheet(ANTLRParser.java:35)
    at com.github.sommeri.less4j.core.ThreadUnsafeLessCompiler.doCompile(ThreadUnsafeLessCompiler.java:58)
    at com.github.sommeri.less4j.core.ThreadUnsafeLessCompiler.compile(ThreadUnsafeLessCompiler.java:47)
    at com.github.sommeri.less4j.core.ThreadUnsafeLessCompiler.compile(ThreadUnsafeLessCompiler.java:29)
    at com.github.sommeri.less4j.core.DefaultLessCompiler.compile(DefaultLessCompiler.java:15)
    at ro.isdc.wro.extensions.processor.css.Less4jProcessor.process(Less4jProcessor.java:56)
    at ro.isdc.wro.model.resource.processor.decorator.ProcessorDecorator.process(ProcessorDecorator.java:87)
    at ro.isdc.wro.model.resource.processor.decorator.LazyProcessorDecorator.process(LazyProcessorDecorator.java:54)
    at ro.isdc.wro.model.resource.processor.decorator.ProcessorDecorator.process(ProcessorDecorator.java:87)
    at ro.isdc.wro.model.resource.processor.decorator.ProcessorDecorator.process(ProcessorDecorator.java:87)
    at ro.isdc.wro.model.resource.processor.decorator.ProcessorDecorator.process(ProcessorDecorator.java:87)
    at ro.isdc.wro.model.resource.processor.decorator.SupportAwareProcessorDecorator.process(SupportAwareProcessorDecorator.java:39)
    at ro.isdc.wro.model.resource.processor.decorator.ProcessorDecorator.process(ProcessorDecorator.java:87)
    at ro.isdc.wro.model.resource.processor.decorator.ExceptionHandlingProcessorDecorator.process(ExceptionHandlingProcessorDecorator.java:56)
    at ro.isdc.wro.model.resource.processor.decorator.ProcessorDecorator.process(ProcessorDecorator.java:87)
    at ro.isdc.wro.model.resource.processor.decorator.BenchmarkProcessorDecorator.process(BenchmarkProcessorDecorator.java:44)
    at ro.isdc.wro.model.resource.processor.decorator.ProcessorDecorator.process(ProcessorDecorator.java:87)
    at ro.isdc.wro.model.resource.processor.decorator.DefaultProcessorDecorator.process(DefaultProcessorDecorator.java:42)
2013-08-06 19:28:07 DEBUG ro.isdc.wro.model.resource.processor.decorator.BenchmarkProcessorDecorator:71 - StopWatch '': running time (millis) = 992

[alexo]

There seems to be a leak in less4j processor. Could you report this issue on less4j github project page?

Thanks, Alex

[dtrunk90]

https://github.com/SomMeri/less4j/issues/157

[SomMeri]

It looks like flags suggested in this stackoverflow answer solved the problem. They allow tomcat to better manage available PermGen.

Background: PermGen is part of memory used to store class loaders, java classes metadata and so on. It runs out of space when less4j tries to create lexer, but that does not necessary means that less4j is the cause. Links to more detailed explanations:

• in-depth: http://frankkieviet.blogspot.ca/2006/10/how-to-fix-dreaded-permgen-space.html
• easy to read: http://javarevisited.blogspot.sk/2012/01/tomcat-javalangoutofmemoryerror-permgen.html

Something is loading and keeping a lot of classes, and it may or may not be less4j. I think it is not less4j, but I can not fully exclude that possibility yet.

[alexo]

Though I'm not sure this is the cause, the less4j processor creates a new instance of less4j compiler for each processing because less4j is not thread safe. If creating an instance of less4j is an expensive operation, I could use a pool to reuse already created instances. Before doing that, I would like to be able to reproduce the issue. A quickstart project or any relevant configuration details woul be helpful.

[SomMeri]

@alexo PermGen does not store instances, it stores class definitions and class loaders. From what I read last two days, it should run out of memory in these cases:

• code-base (including all frameworks and libraries) is huge (explanation),
• a class created by one class loader is referenced from class created by another class loader and application have been redeployed multiple times (explanation).

Creating another instance of the same class again should not take more of its space, so I do not think reusal of instances would help much. Leaks happen when you deploy and undeploy repeatedly. He configured huge perm gen space (2048m), so class loaders dependency issue is probable.

An example project with all dependencies that cause the issue would help to find responsible class or lib. I do not see how it could be possible to find the leak otherwise.

[alexo]

@dtrunk90 have you managed to reproduce this issue with the latest version of less4j ?

[dtrunk90]

Not tested, I'm using lessCss.less preprocessor now.

[alexo]

I'm closing this issue, since it is related to less4j and probably was fixed in one of the latest version of less4j.