search

manifold-systems / manifold

Manifold is a Java compiler plugin, its features include Metaprogramming, Properties, Extension Methods, Operator Overloading, Templates, a Preprocessor, and more.
http://manifold.systems/
Apache License 2.0
2.43k stars 125 forks source link

Java Compiler in the IntelliJ IDEA costs tripple memory while rebuild with -Xplugin:Manifold. #433

Closed jearton closed 1 year ago

jearton commented 1 year ago

Describe the bug Without manifold, the compiler build process is normal.

image

image

With manifold comes into effect, the compiler build process takes a lot of memory which continuously risen. Finally lead to out of memory.

image

To Reproduce Steps to reproduce the behavior:

  1. Prepare a Java project with 200,000 lines of code and import IntelliJ IDEA

  2. Add manifold dependency

    
    <dependency>
        <groupId>systems.manifold</groupId>
        <artifactId>manifold-ext</artifactId>
       <version>2023.1.3</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>systems.manifold</groupId>
        <artifactId>manifold-ext-rt</artifactId>
       <version>2023.1.3</version>
    </dependency>

  3. Add compiler args

    <build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <compilerArgs>
                    <!--Add the Manifold plugin-->
                    <arg>-Xplugin:Manifold no-bootstrap</arg>
                </compilerArgs>
            </configuration>
        </plugin>
    </plugins>
</build>

  4. rebuild the project.

Expected behavior

The build process should cost memory less than 1G.

I had to give it up because of the huge gap between using manifold or not in memory usage.

Desktop (please complete the following information):

Stack trace

java: java.lang.OutOfMemoryError: GC overhead limit exceeded
java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at com.sun.tools.javac.main.Main.compile(Main.java:559)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
    at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
    at org.jetbrains.jps.javac.JavacMain.compile(JavacMain.java:237)
    at org.jetbrains.jps.javac.ExternalJavacProcess.compile(ExternalJavacProcess.java:196)
    at org.jetbrains.jps.javac.ExternalJavacProcess.access$400(ExternalJavacProcess.java:30)
    at org.jetbrains.jps.javac.ExternalJavacProcess$CompilationRequestsHandler$1.run(ExternalJavacProcess.java:269)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at manifold.api.fs.jar.JarEntryDirectoryImpl.getOrCreateDirectory(JarEntryDirectoryImpl.java:47)
    at manifold.api.fs.jar.JarFileDirectoryImpl.processJarEntry(JarFileDirectoryImpl.java:124)
    at manifold.api.fs.jar.JarFileDirectoryImpl.<init>(JarFileDirectoryImpl.java:67)
    at manifold.api.fs.def.FileSystemImpl.createDir(FileSystemImpl.java:177)
    at manifold.api.fs.def.FileSystemImpl.getIDirectory(FileSystemImpl.java:85)
    at manifold.internal.host.JavacManifoldHost.lambda$initPaths$2(JavacManifoldHost.java:76)
    at manifold.internal.host.JavacManifoldHost$$Lambda$611/1308774127.apply(Unknown Source)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
    at manifold.internal.host.JavacManifoldHost.initPaths(JavacManifoldHost.java:76)
    at manifold.internal.host.JavacManifoldHost.initialize(JavacManifoldHost.java:70)
    at manifold.internal.javac.JavacPlugin.initialize(JavacPlugin.java:1008)
    at manifold.internal.javac.JavacPlugin.started(JavacPlugin.java:1041)
    at com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.started(ClientCodeWrapper.java:668)
    at com.sun.tools.javac.api.MultiTaskListener.started(MultiTaskListener.java:103)
    at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:978)
    at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857)
    at com.sun.tools.javac.main.Main.compile(Main.java:523)
    ... 9 more
rsmckinney commented 1 year ago

Hi. Yes, the manifold plugin does incur memory overhead, the amount of which varies depending on project structure, class hierarchy, etc. 3x is definitely on the high end. This will be addressed in future release. Apologies, for the inconvenience.