Very large memory usage in a multimodule build with gradle & scalafmt

[SheliakLyr]

I have a very large multi-module project build by gradle. Recently, I have added a spotless plugin to some of the modules (let's say 20 out of a few hundreds).

With spotless plugin applied, the build uses much more memory. My current memory setting is -Xmx3700m for gradle daemon and the build often fails due to OOM exception. Checking the memory snapshot, I can see that the biggest object is the com.diffplug.gradle.spotless.JvmLocalCache. It retains (keeps in memory) almost 1.5GB of data.

I cannot provide you with a snapshot, but I have discovered, that this cache holds some questionable objects, like DefaultResolverStatus'es. I think, that scalafmtStep might be responsible for inserting those large objects into the cache:

|                          +---[10] of java.util.HashMap$Node[2048]                                                                                                                                  |         34,320  |         8,208  |
|                            |                                                                                                                                                                       |                 |                |
|                            +---table of java.util.HashMap size = 816                                                                                                                               |         34,368  |            48  |
|                              |                                                                                                                                                                     |                 |                |
|                              +---projects of org.gradle.api.internal.project.DefaultProjectRegistry                                                                                                |        267,032  |            24  |
|                                |                                                                                                                                                                   |                 |                |
|                                +---projectRegistry of org.gradle.api.internal.project.DefaultCrossProjectModelAccess                                                                               |             16  |            16  |
|                                  |                                                                                                                                                                 |                 |                |
|                                  +---__crossProjectModelAccess__ of org.gradle.api.internal.project.DefaultProject_Decorated                                                                       |         36,904  |           232  |
|                                    |                                                                                                                                                               |                 |                |
|                                    +---arg$3 of com.diffplug.gradle.spotless.GradleProvisioner$$Lambda$1145                                                                                        |             24  |            24  |
|                                      |                                                                                                                                                             |                 |                |
|                                      +---provisioner of com.diffplug.gradle.spotless.GradleProvisioner$DedupingProvisioner                                                                         |            112  |            24  |
|                                        |                                                                                                                                                           |                 |                |
|                                        +---arg$2 of com.diffplug.spotless.scala.ScalaFmtStep$$Lambda$1147                                                                                          |            168  |            24  |
|                                          |                                                                                                                                                         |                 |                |
|                                          +---stateSupplier of com.diffplug.spotless.FormatterStepImpl$Standard                                                                                     |            200  |            32  |
|                                            |                                                                                                                                                       |                 |                |
|                                            +---[0] of java.lang.Object[10]                                                                                                                         |            256  |            56  |
|                                              |                                                                                                                                                     |                 |                |
|                                              +---elementData of java.util.ArrayList size = 1                                                                                                       |            280  |            24  |
|                                                |                                                                                                                                                   |                 |                |
|                                                +---value of java.util.HashMap$Node (JvmLocalCache$InternalCacheKey → ArrayList) chunk size = 2                                                     |            152  |            32  |
|                                                  |                                                                                                                                                 |                 |                |
|                                                  +---[3] of java.util.HashMap$Node[64]                                                                                                             |  1,439,569,344  |           272  |
|                                                    |                                                                                                                                               |                 |                |
|                                                    +---table of java.util.HashMap size = 40                                                                                                        |  1,439,569,392  |            48  |
|                                                      |                                                                                                                                             |                 |                |
|                                                      +---m of java.util.Collections$SynchronizedMap size = 40                                                                                      |  1,439,569,424  |            32  |
|                                                        |                                                                                                                                           |                 |                |
|                                                        +---daemonState of com.diffplug.gradle.spotless.JvmLocalCache org.gradle.internal.classloader.VisitableURLClassLoader (2)                   |  1,439,569,536  |            80  |
|                                                          |                                                                                                                                         |                 |                |
|                                                          +---[274] of java.lang.Object[320]                                                                                                        |  1,439,594,792  |         1,296  |
|                                                            |                                                                                                                                       |                 |                |
|                                                            +---elementData of java.util.Vector size = 316                                                                                          |  1,439,594,824  |            32  |
|                                                              |                                                                                                                                     |                 |                |
|                                                              +---classes of org.gradle.internal.classloader.VisitableURLClassLoader (2)                                                            |  1,439,868,360  |            88  |
|                                                                |                                                                                                                                   |                 |                |
|                                                                +---parent of org.gradle.groovy.scripts.internal.DefaultScriptCompilationHandler$ScriptClassLoader (1610)                           |          2,512  |            96  |
|                                                                  |                                                                                                                                 |                 |                |
|                                                                  +---classloader of java.security.ProtectionDomain                                                                                 |            592  |            40  |
|                                                                    |                                                                                                                               |                 |                |
|                                                                    +---[8] of java.security.ProtectionDomain[16]                                                                                   |             80  |            80  |
|                                                                      |                                                                                                                             |                 |                |
|                                                                      +---context of java.security.AccessControlContext                                                                             |            120  |            40  |
|                                                                        |                                                                                                                           |                 |                |
|                                                                        +---inheritedAccessControlContext of java.lang.Thread [Thread, Stack Local] "JGit-WorkQueue" daemon tid=92 [TIMED_WAITING]  |            360  |           120  |

Gradle version: 7.4.0 Gradle spotless plugin: 6.3.0 Operating system: Arch, Ubuntu

subprojects {
  apply plugin: "com.diffplug.spotless"

  spotless {
    scala {
      scalafmt("3.0.7").configFile(rootProject.file('.scalafmt.conf'))
    }
  }
}

[nedtwigg]

JvmLocalCache is a workaround we added in 6.0 to make the configuration cache work. We described the reasoning around this in #987.

There's two main workarounds:

• try using lineEndings 'UNIX' (this will put less data into the cache than the default value GIT_ATTRIBUTES)
• downgrade your Spotless to 5.17.1 - you'll lose support for configuration cache, but you'll get better memory consumption (with this downgrade, the line endings won't matter anymore)

I'd be curious to see how much either of the downgrades above help for your case.

[SheliakLyr]

Thank you for the tips @nedtwigg. I did some tests but right now I do not have any conclusive results, I stopped being able to OOM the build. We might have introduced some other change in the meantime, or maybe I need to run the build multiple times with some specific changes... can't say.

Right now, JvmLocalCache takes 400MB of memory after running spotlessApply. Changing 'lineEndings' does not seem to do much - most of the memory is kept in ":steps" attributes.

Downgrading spotless of course removes JvmLocalCache, so comparisions are more difficult, but the memory of the entire process stays roughly the same.

For now, I will set lineEndings to UNIX (I had to do it for other reasons as well) and see how the build behaves. I guess this issue can be closed for now, I can open a new one if I find a real cause of the problem.

[nedtwigg]

Downgrading spotless... the memory of the entire process stays roughly the same

That's good news! When we were building JvmLocalCache, we knew there was a risk of being too memory hungry, and we did do some work to slim it down.

Thanks for opening this issue, and happy to revisit if anything new comes to light.