search

iTwin / imodel-native

Apache License 2.0
20 stars 12 forks source link

Android builds sometimes fail with apparent OOM #418

Open pmconne opened 1 year ago

pmconne commented 1 year ago

Execution failed for task ':app:mergeReleaseNativeDebugMetadata'.

A failure occurred while executing com.android.build.gradle.internal.tasks.MergeNativeDebugMetadataTask$MergeNativeDebugMetadataWorkAction Java heap space

tcobbs-bentley commented 1 year ago

For reference, here is a link to a failed pipeline run:

https://dev.azure.com/bentleycs/iModelTechnologies/_build/results?buildId=2340491&view=logs&jobId=023ba795-95cf-53ec-173c-c0bcaae2690b&j=023ba795-95cf-53ec-173c-c0bcaae2690b&t=712cbd68-614e-5470-00b0-5caeb3baa3b2

tcobbs-bentley commented 1 year ago

I just found the following:

GRADLE_OPTS Specifies JVM arguments to use when starting the Gradle client VM. The client VM only handles command line input/output, so it is rare that one would need to change its VM options. The actual build is run by the Gradle daemon, which is not affected by this environment variable.

(See here.)

That same page includes the following:

org.gradle.jvmargs=(JVM arguments) Specifies the JVM arguments used for the Gradle Daemon. The setting is particularly useful for configuring JVM memory settings for build performance. This does not affect the JVM settings for the Gradle client VM. The default is -Xmx512m "-XX:MaxMetaspaceSize=384m".

Between these two statements, it certainly seems that setting GRADLE_OPTS like we do in our yml script has no effect on the build memory. I know that display-test-app has the following in its gradle.properties:

org.gradle.jvmargs=-Xmx4096m -Dfile.encoding=UTF-8

The web page does in another section say that JAVA_OPTS can be used to configure the VM memory settings when the Gradle Daemon is deactivated. Since our command line does include --no-daemon, does that perhaps mean that GRADLE_OPTS will work?

@swbsi, what do you think?

tcobbs-bentley commented 1 year ago

I also see that -Xms sets the initial pool size, while -Xmx sets the max. So if the problem is that the machine doesn't have enough virtual memory available, omitting -Xms would fix that as long as it has enough virtual memory as is actually needed.

tcobbs-bentley commented 1 year ago

I created a PR in bsicommon to only set the max heap size, not the initial heap size (going on the assumption that the GRADLE_OPTS environment variable does in fact do what we want when using --no-daemon). I'm not sure how to test the change.

pmconne commented 1 year ago

@tcobbs-bentley can you link to that PR? Are you still blocked? This is still breaking builds.

tcobbs-bentley commented 1 year ago

@pmconne The PR is here:

https://dev.azure.com/bentleycs/iModelTechnologies/_git/bsicommon-imt/pullrequest/360429

Since we weren't sure whether we weren't allocating enough or requiring too much, that PR simply got rid of the minimum allocation. I'll submit another PR to increase the max allocation.

tcobbs-bentley commented 1 year ago

Here is a PR to double Gradle's maximum heap size to 8GB:

https://dev.azure.com/bentleycs/iModelTechnologies/_git/bsicommon-imt/pullrequest/367245

tcobbs-bentley commented 1 year ago

@pmconne Has it failed since my last PR above?

pmconne commented 1 year ago

@pmconne Has it failed since my last PR above?

Not that I have observed.