Occasional OOMEs when running the test suite

[stefanvodita]

Description

I've seen OutOfMemoryErrors a few times recently when doing ./gradlew test or ./gradlew check. I don't think this is related to my changes, because it's intermittent and I've seen it across different branches. I tried reporting it at https://bugreport.java.com, but I didn't get a response or an issue to track. Maybe someone here has some insight.

> Task :lucene:backward-codecs:compileTestJava
An exception has occurred in the compiler (21.0.1). Please file a bug against the Java compiler via the Java bug reporting page (https://bugreport.java.com) after checking the Bug Database (https://bugs.java.com) for duplicates. Include your program, the following diagnostic, and the parameters passed to the Java compiler in your report. Thank you.
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at jdk.compiler/com.sun.tools.javac.comp.Check$$Lambda/0x0000009002630470.get(Unknown Source)
        at java.base/java.util.stream.Collectors.lambda$groupingBy$52(Collectors.java:1106)
        at java.base/java.util.stream.Collectors$$Lambda/0x00000090016e4328.apply(Unknown Source)
        at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1228)
        at java.base/java.util.stream.Collectors.lambda$groupingBy$53(Collectors.java:1106)
        at java.base/java.util.stream.Collectors$$Lambda/0x00000090016d8c28.accept(Unknown Source)
        at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at jdk.compiler/com.sun.tools.javac.comp.Check.methodsGroupedByName(Check.java:2879)
        at jdk.compiler/com.sun.tools.javac.comp.Check.checkPotentiallyAmbiguousOverloads(Check.java:2754)
        at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:5628)
        at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClass(Attr.java:5543)
        at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClass(Attr.java:5367)
        at jdk.compiler/com.sun.tools.javac.comp.Attr.attrib(Attr.java:5306)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1359)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:976)
        at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:104)
        at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl$$Lambda/0x0000009001a5e478.call(Unknown Source)
        at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:152)
        at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
        at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
        at org.gradle.internal.compiler.java.IncrementalCompileTask.call(IncrementalCompileTask.java:92)
        at org.gradle.api.internal.tasks.compile.AnnotationProcessingCompileTask.call(AnnotationProcessingCompileTask.java:94)
        at org.gradle.api.internal.tasks.compile.ResourceCleaningCompilationTask.call(ResourceCleaningCompilationTask.java:57)
        at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:55)

Version and environment details

No response

[dweiss]

This looks like the garbage collector (gradle's JVM) got close to the heap limit and it suffocated trying to release memory while other threads kept allocating it (GC overhead limit exceeded).

There's a lot going on in gradle's JVM, especially with parallel builds... and some of it is system-dependent (task parallelism, gc speed). I suggest we can keep this open and see how frequently this happens. Could be a gradle bug (memory leak somewhere), could be the heap that's too small for the number of tasks running in parallel. Hard to tell.