search

github / codeql-action

Actions for running CodeQL analysis
MIT License
1.17k stars 327 forks source link

OutOfMemoryError: GC overhead limit exceeded during autobuild #2604

Open mudasar187 opened 1 week ago

mudasar187 commented 1 week ago 
[2024-11-13 08:18:41] [autobuild] e: java.lang.OutOfMemoryError: GC overhead limit exceeded
  [2024-11-13 08:18:41] [autobuild]     at java.base/jdk.internal.org.objectweb.asm.SymbolTable.<init>(SymbolTable.java:156)
  [2024-11-13 08:18:41] [autobuild]     at java.base/jdk.internal.org.objectweb.asm.ClassWriter.<init>(ClassWriter.java:297)
  [2024-11-13 08:18:41] [autobuild]     at java.base/jdk.internal.org.objectweb.asm.ClassWriter.<init>(ClassWriter.java:267)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.InnerClassLambdaMetafactory.<init>(InnerClassLambdaMetafactory.java:185)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:336)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.LambdaForm$DMH/0x00007f134c091400.invokeStatic(LambdaForm$DMH)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:134)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:316)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:274)
  [2024-11-13 08:18:41] [autobuild]     at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:264)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializationJvmIrIntrinsicSupport.<init>(SerializationJvmIrIntrinsicSupport.kt:172)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlinx.serialization.compiler.extensions.SerializationLoweringExtension.getPlatformIntrinsicExtension(SerializationLoweringExtension.kt:200)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.backend.jvm.JvmIrCodegenFactory.invokeLowerings(JvmIrCodegenFactory.kt:345)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.codegen.CodegenFactory.generateModule(CodegenFactory.kt:46)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.backend.jvm.JvmIrCodegenFactory.generateModuleInFrontendIRMode(JvmIrCodegenFactory.kt:433)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.cli.jvm.compiler.pipeline.JvmCompilerPipelineKt.generateCodeFromIr(jvmCompilerPipeline.kt:246)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.cli.jvm.compiler.pipeline.JvmCompilerPipelineKt.compileModulesUsingFrontendIrAndLightTree(jvmCompilerPipeline.kt:142)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:148)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:43)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:103)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:49)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.runCompiler(IncrementalJvmCompilerRunner.kt:464)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.runCompiler(IncrementalJvmCompilerRunner.kt:73)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.doCompile(IncrementalCompilerRunner.kt:506)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compileImpl(IncrementalCompilerRunner.kt:423)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compileNonIncrementally(IncrementalCompilerRunner.kt:301)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compile(IncrementalCompilerRunner.kt:129)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.daemon.CompileServiceImplBase.execIncrementalCompiler(CompileServiceImpl.kt:675)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.daemon.CompileServiceImplBase.access$execIncrementalCompiler(CompileServiceImpl.kt:92)
  [2024-11-13 08:18:41] [autobuild]     at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1660)
  [2024-11-13 08:18:41] [autobuild] FAILURE: Build failed with an exception.
  [2024-11-13 08:18:41] [autobuild] * What went wrong:
  [2024-11-13 08:18:41] [autobuild] Execution failed for task ':compileTestKotlin'.
  [2024-11-13 08:18:41] [autobuild] > A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
  [2024-11-13 08:18:41] [autobuild]    > Not enough memory to run compilation. Try to increase it via 'gradle.properties':
  [2024-11-13 08:18:41] [autobuild]      kotlin.daemon.jvmargs=-Xmx<size>
  [2024-11-13 08:18:41] [autobuild] * Try:
  [2024-11-13 08:18:41] [autobuild] > Run with --info or --debug option to get more log output.
  [2024-11-13 08:18:41] [autobuild] > Run with --scan to get full insights.
  [2024-11-13 08:18:41] [autobuild] > Get more help at https://help.gradle.org.
  [2024-11-13 08:18:41] [autobuild] * Exception is:
  [2024-11-13 08:18:41] [autobuild] org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':compileTestKotlin'.
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:130)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:293)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:128)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:116)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:74)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
  [2024-11-13 08:18:41] [autobuild]     at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)

Steps to Reproduce:
Trigger the autobuild process. Observe the build failure due to OutOfMemoryError.

Expected Behavior: The build should complete successfully without running out of memory.

Actual Behavior: The build fails with an OutOfMemoryError.

Suggested Fix: Increase the memory allocation for the Kotlin daemon in gradle.properties: kotlin.daemon.jvmargs=-Xmx<size>

Environment:
Kotlin version: 2.0.21 Gradle version: 8.11 Java version: 21

smowton commented 1 week ago

Can you give an example of a project where you're seeing this OOM condition?

mudasar187 commented 4 days ago

@smowton https://github.com/navikt/sokos-oppdrag/actions/runs/11886775839/job/33118460142#step:4:118

smowton commented 4 days ago

Having tried this locally, I find the max memory pressure from that job is around 3.5GB, which should be achievable on a standard GHA runner. Have you tried the gradle.properties fix suggested?

mudasar187 commented 3 days ago

Yeah i have tried. But it seems wierd, because I have another project which using more memory, but I did not needed to make any fix as this error suggested. This project using less memory I needed to set 4 gb memory then it ran successfully. Is that something you will look at ? Or.. ?

smowton commented 3 days ago

Because our current Kotlin extractor runs as a plugin to the Kotlin compiler which is itself managed by Gradle, unfortunately for now we need such memory-usage matters to be addressed at a higher level that isn't in our direct control. We anticipate in the future Kotlin extraction will likely use a different process model at which point we will have greater control over our own operating environment; however, for now I'm afraid it will occasionally be necessary to give Gradle more memory.