search

scalameta / metals

Scala language server with rich IDE features 🚀
https://scalameta.org/metals/
Apache License 2.0
2.07k stars 325 forks source link

Exception in thread java.lang.OutOfMemoryError: Java heap space #5589

Open olatheander opened 1 year ago

olatheander commented 1 year ago

Describe the bug

I'm opening a fairly large Scala project using neovim & nvim-metals and when opening a source file most of the time (but not always) the metals process gets stuck in high CPU and memory usage: image

The status report (metals_status) typically reads Compiling myExample 8m20s (0%%) and no progress is made.

Restarting metals either via MetalsRestartMetals command or by killing the process doesn't seem to help, it goes straight up to high CPU and high memory usage again: image

Checking the metals.log there are a bunch of OOMs:

Exception in thread "pool-2-thread-158" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:374)
        at com.google.common.hash.AbstractStreamingHasher.<init>(AbstractStreamingHasher.java:68)
        at com.google.common.hash.AbstractStreamingHasher.<init>(AbstractStreamingHasher.java:50)
        at com.google.common.hash.Murmur3_128HashFunction$Murmur3_128Hasher.<init>(Murmur3_128HashFunction.java:96)
        at com.google.common.hash.Murmur3_128HashFunction.newHasher(Murmur3_128HashFunction.java:65)
        at com.google.common.hash.AbstractHashFunction.hashObject(AbstractHashFunction.java:36)
        at com.google.common.hash.BloomFilterStrategies$2.put(BloomFilterStrategies.java:109)
        at com.google.common.hash.BloomFilter.put(BloomFilter.java:176)
        at scala.meta.internal.metals.ReferenceProvider.$anonfun$onChange$5(ReferenceProvider.scala:79)
        at scala.meta.internal.metals.ReferenceProvider.$anonfun$onChange$5$adapted(ReferenceProvider.scala:75)
        at scala.meta.internal.metals.ReferenceProvider$$Lambda$4396/0x00007ff174c4b150.apply(Unknown Source)
        at scala.collection.immutable.VectorStatics$.foreachRec(Vector.scala:2122)
        at scala.collection.immutable.Vector.foreach(Vector.scala:2128)
        at scala.meta.internal.metals.ReferenceProvider.$anonfun$onChange$4(ReferenceProvider.scala:75)
        at scala.meta.internal.metals.ReferenceProvider.$anonfun$onChange$4$adapted(ReferenceProvider.scala:74)
        at scala.meta.internal.metals.ReferenceProvider$$Lambda$4395/0x00007ff174c4ad90.apply(Unknown Source)
        at scala.collection.immutable.Vector.foreach(Vector.scala:2122)
        at scala.meta.internal.metals.ReferenceProvider.$anonfun$onChange$1(ReferenceProvider.scala:74)
        at scala.meta.internal.metals.ReferenceProvider.$anonfun$onChange$1$adapted(ReferenceProvider.scala:63)
        at scala.meta.internal.metals.ReferenceProvider$$Lambda$4386/0x00007ff174c483c0.apply(Unknown Source)
        at scala.Option.map(Option.scala:242)
        at scala.meta.internal.metals.ReferenceProvider.onChange(ReferenceProvider.scala:63)
        at scala.meta.internal.metals.SemanticdbIndexer.$anonfun$onChange$1(SemanticdbIndexer.scala:85)
        at scala.meta.internal.metals.SemanticdbIndexer.$anonfun$onChange$1$adapted(SemanticdbIndexer.scala:85)
        at scala.meta.internal.metals.SemanticdbIndexer$$Lambda$4383/0x00007ff174c47908.apply(Unknown Source)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:85)
        at scala.meta.internal.metals.SemanticdbIndexer.$anonfun$onChange$2(SemanticdbIndexer.scala:92)
        at scala.meta.internal.metals.SemanticdbIndexer.$anonfun$onChange$2$adapted(SemanticdbIndexer.scala:91)
        at scala.meta.internal.metals.SemanticdbIndexer$$Lambda$4381/0x00007ff174c47188.apply(Unknown Source)
        at scala.Option.foreach(Option.scala:437)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:91)
Exception in thread "pool-2-thread-141" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-170" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-69" java.lang.OutOfMemoryError: Java heap space
        at scala.meta.internal.semanticdb.SymbolOccurrence$.parseFrom(SymbolOccurrence.scala:118)
        at scala.meta.internal.semanticdb.SymbolOccurrence$.parseFrom(SymbolOccurrence.scala:98)
        at scalapb.LiteParser$.readMessage(LiteParser.scala:24)
        at scala.meta.internal.semanticdb.TextDocument$.parseFrom(TextDocument.scala:240)
        at scala.meta.internal.semanticdb.TextDocument$.parseFrom(TextDocument.scala:210)
        at scalapb.LiteParser$.readMessage(LiteParser.scala:24)
        at scala.meta.internal.semanticdb.TextDocuments$.parseFrom(TextDocuments.scala:69)
        at scala.meta.internal.semanticdb.TextDocuments$.parseFrom(TextDocuments.scala:59)
        at scalapb.GeneratedMessageCompanion.parseFrom(GeneratedMessageCompanion.scala:186)
        at scalapb.GeneratedMessageCompanion.parseFrom$(GeneratedMessageCompanion.scala:186)
        at scala.meta.internal.semanticdb.TextDocuments$.parseFrom(TextDocuments.scala:59)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:90)
        at scala.meta.internal.metals.MetalsLspService.$anonfun$didChangeWatchedFiles$7(MetalsLspService.scala:1241)
        at scala.meta.internal.metals.MetalsLspService$$Lambda$4204/0x00007ff174c0ca38.apply$mcV$sp(Unknown Source)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
        at scala.concurrent.Future$$$Lambda$987/0x00007ff174378560.apply(Unknown Source)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.runWith(Thread.java:1636)
        at java.base/java.lang.Thread.run(Thread.java:1623)
Exception in thread "pool-2-thread-258" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-86" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-47" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-2" java.lang.OutOfMemoryError: Java heap space

Doing a jmap -histo <PID> for the metals process when OOM:ing:

 num     #instances         #bytes  class name (module)
-------------------------------------------------------
   1:      33902871     5554695840  [B (java.base@20.0.2)
   2:      33809316      811423584  java.lang.String (java.base@20.0.2)
   3:       5998191      335898696  scala.meta.internal.semanticdb.SymbolInformation
   4:      10250481      328015392  scala.meta.internal.semanticdb.TypeRef
   5:       6675126      221817488  [Ljava.lang.Object; (java.base@20.0.2)
   6:       6753244      216103808  scala.meta.internal.semanticdb.Range
   7:       6675597      213619104  scala.meta.internal.semanticdb.SymbolOccurrence
   8:       4842706      116224944  scala.meta.internal.semanticdb.ValueSignature
   9:       7208864      115341824  scala.Some
  10:       6219994       99519904  scala.collection.immutable.Vector1
  11:       2438209       39011344  scala.meta.internal.semanticdb.PublicAccess
  12:       1440902       34581648  scala.meta.internal.semanticdb.Annotation
  13:       1314698       31552752  scala.meta.internal.semanticdb.AnnotatedType
  14:       1180665       28335960  scala.meta.internal.semanticdb.Scope
  15:       1121238       26909712  scala.meta.internal.semanticdb.ByNameType
  16:        745589       23858848  scala.meta.internal.semanticdb.MethodSignature
  17:          7913       10836120  Ljdk.internal.vm.FillerArray; (java.base@20.0.2)
  18:         11306        9926840  [J (java.base@20.0.2)
  19:        207233        6631456  scala.meta.internal.semanticdb.TypeSignature
  20:        202674        6485568  scala.meta.internal.semanticdb.ClassSignature
  21:        127463        3059112  scala.meta.internal.semanticdb.ExistentialType
  22:        126017        3024408  scala.meta.internal.semanticdb.SingleType
  23:        125957        3022968  scala.meta.internal.semanticdb.ConstantType
  24:        122957        2950968  scala.meta.internal.semanticdb.IntConstant
  25:         58288        2331520  java.util.LinkedHashMap$Entry (java.base@20.0.2)

It seems like semanticdb gets a bit carried away...

Expected behavior

No response

Operating system

Linux

Editor/Extension

Nvim (nvim-metals)

Version of Metals

v1.0.0

Extra context or search terms

No response

tgodzik commented 1 year ago

Thanks for reporting! Does changing the -Xmx setting help in any way? I think we default to 1gb which might not be enough in large codebases.

The oom happens during indexing generated semanticdb files, but I haven't seen it being an issue most of the time. Not sure if we can improve anything there unless it's some kind of memory leak? But I don't see any indications of it

olatheander commented 1 year ago

@tgodzik a bit embarrassing but I haven't managed to figure out how to configure the -Xmx setting for metals in my tool-chain: nvim -> nvim-metals -> metals. Any pointer in the right direction would be appreciated.

As a side note, I did open the metals project itself, since that's an even larger project than the one I'm working on and so far it works fine so maybe there is something very specific in that code-base that makes it go OOM.

olatheander commented 1 year ago

OK, I think I bumped the heap-size correctly, added serverProperties to the neovim config as:

metals_config.settings = {
    showImplicitArguments = true,
    excludedPackages = { "akka.actor.typed.javadsl", "com.github.swagger.akka.javadsl" },
    serverProperties = { "-Xmx3G" }
}

and it seems like it helped. No OOMs so far (albeit I only tested it for a short time).

tgodzik commented 1 year ago

Let us know if you encounter more OOMs, I am highly surprised that it went OOM on a smaller project than metals

olatheander commented 1 year ago

Seems like I celebrated a bit early, new OOMs appears but seems to be a different stack-trace:

Flag -Ximport-suggestion-timeout set repeatedly
Exception in thread "pool-2-thread-177" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.nio.file.Files.read(Files.java:3213)
        at java.base/java.nio.file.Files.readAllBytes(Files.java:3270)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:90)
        at scala.meta.internal.metals.MetalsLspService.$anonfun$didChangeWatchedFiles$7(MetalsLspService.scala:1241)
        at scala.meta.internal.metals.MetalsLspService$$Lambda$3080/0x00007f84f482f2e8.apply$mcV$sp(Unknown Source)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
        at scala.concurrent.Future$$$Lambda$987/0x00007f84f4378560.apply(Unknown Source)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.runWith(Thread.java:1636)
        at java.base/java.lang.Thread.run(Thread.java:1623)
Exception in thread "pool-2-thread-171" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.nio.file.Files.read(Files.java:3213)
        at java.base/java.nio.file.Files.readAllBytes(Files.java:3270)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:90)
        at scala.meta.internal.metals.MetalsLspService.$anonfun$didChangeWatchedFiles$7(MetalsLspService.scala:1241)
        at scala.meta.internal.metals.MetalsLspService$$Lambda$3080/0x00007f84f482f2e8.apply$mcV$sp(Unknown Source)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
        at scala.concurrent.Future$$$Lambda$987/0x00007f84f4378560.apply(Unknown Source)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.runWith(Thread.java:1636)
        at java.base/java.lang.Thread.run(Thread.java:1623)
Exception in thread "pool-2-thread-175" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.nio.file.Files.read(Files.java:3213)
        at java.base/java.nio.file.Files.readAllBytes(Files.java:3270)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:90)
        at scala.meta.internal.metals.MetalsLspService.$anonfun$didChangeWatchedFiles$7(MetalsLspService.scala:1241)
        at scala.meta.internal.metals.MetalsLspService$$Lambda$3080/0x00007f84f482f2e8.apply$mcV$sp(Unknown Source)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
        at scala.concurrent.Future$$$Lambda$987/0x00007f84f4378560.apply(Unknown Source)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.runWith(Thread.java:1636)
        at java.base/java.lang.Thread.run(Thread.java:1623)
Exception in thread "pool-2-thread-178" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.nio.file.Files.read(Files.java:3213)
        at java.base/java.nio.file.Files.readAllBytes(Files.java:3270)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:90)
        at scala.meta.internal.metals.MetalsLspService.$anonfun$didChangeWatchedFiles$7(MetalsLspService.scala:1241)
        at scala.meta.internal.metals.MetalsLspService$$Lambda$3080/0x00007f84f482f2e8.apply$mcV$sp(Unknown Source)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
        at scala.concurrent.Future$$$Lambda$987/0x00007f84f4378560.apply(Unknown Source)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.runWith(Thread.java:1636)
        at java.base/java.lang.Thread.run(Thread.java:1623)
Exception in thread "pool-2-thread-174" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-180" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-144" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-121" java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects
...
..
Exception in thread "pool-2-thread-173" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-150" java.lang.OutOfMemoryError: Java heap space
aug. 29, 2023 10:06:51 FM scala.meta.internal.pc.CompilerAccess handleError
SEVERE: A severe compiler error occurred, full details of the error can be found in the error report /home/************/.metals/.reports/metals-full/r_compiler-error_1693296384954
Flag -Ximport-suggestion-timeout set repeatedly
...
..

The beginning of the report r_compiler-error_1693296384954 reads:

java.lang.OutOfMemoryError: Java heap space
occurred in the presentation compiler.

action parameters:
offset: 10079
uri: file://<WORKSPACE>/...../MyFile.scala
text:
...
** the source of the file **
..
error stacktrace:

After my config change above metals executes with a larger heap space: java -Xss4m -Xms100m -Xmx3G -jar .../metals. I will try and bump the heap size a bit more to see if that changes anything.

tgodzik commented 1 year ago

Looks like something weird is happening with semanticdb files. They are being read and the memory never freed? Anything specific about that project? Or is it happening with all the projects?

olatheander commented 1 year ago

I just recently configured neovim for Scala so I haven't tried that many different projects; basically just this particular project, the metals project and some very small hello-world:ish projects. It seems to be only this project that OOM. But as far as I can tell there is nothing special about the project and opening it in e.g. IntelliJ (which I have tried too) it works just fine.

I also tried bumping to -Xmx5G but it still OOMs.

Can I enable some additional logging etc to perhaps provide more info in the metal's log?

tgodzik commented 1 year ago

Adding -Dmetals.loglevels=debug might help or -Dmetals.statistics=all :thinking:

olatheander commented 1 year ago

I'll try that.

FYI, I did try to set the a ridiculously high value -Xmx16G and it seems to help. No OOMs so far, thus it seems like at least it's not runaway allocation going on.

olatheander commented 1 year ago

Did try to add

    serverProperties = { "-Xmx5G", "-Dmetals.loglevels=debug", "-Dmetals.statistics=all" }

but it didn't affect the logging in shown by MetalsToggleLogs, only INFO and above are logged. Does the debug logs end up elsewhere perhaps?

tgodzik commented 1 year ago

My bad, it's actually -Dmetals.loglevel sorry! Not sure where that s snuck in. Though the statistics, should be shown :thinking:

olatheander commented 1 year ago

Now it worked. I got a bunch of these in the log:

2023.09.04 17:48:35 DEBUG SemanticdbPath(<project-root>/.bloop/schema/bloop-bsp-clients-classes/classes-Metals-PwrQptAzRAGT7JKkPNO3sg==/META-INF/semanticdb/schema/target/scala-3.2.2/src_managed/main/.../.../SomeClass.scala.semanticdb) is not yet ready
com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field.  This could mean either that the input has been truncated or that an embedded message misreported its own length.
        at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:115)
        at com.google.protobuf.CodedInputStream$ArrayDecoder.pushLimit(CodedInputStream.java:1190)
        at scalapb.LiteParser$.readMessage(LiteParser.scala:23)
        at scala.meta.internal.semanticdb.TextDocuments$.parseFrom(TextDocuments.scala:69)
        at scala.meta.internal.semanticdb.TextDocuments$.parseFrom(TextDocuments.scala:59)
        at scalapb.GeneratedMessageCompanion.parseFrom(GeneratedMessageCompanion.scala:186)
        at scalapb.GeneratedMessageCompanion.parseFrom$(GeneratedMessageCompanion.scala:186)
        at scala.meta.internal.semanticdb.TextDocuments$.parseFrom(TextDocuments.scala:59)
        at scala.meta.internal.metals.SemanticdbIndexer.onChange(SemanticdbIndexer.scala:90)
        at scala.meta.internal.metals.MetalsLspService.$anonfun$didChangeWatchedFiles$7(MetalsLspService.scala:1273)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.lang.Thread.run(Thread.java:1623)
tgodzik commented 1 year ago

That's not an issue, seems like we try to read it and fail, but we retry later on.

shotexa commented 1 year ago

I'm having the same issue, in my case in VSCode, here are the logs

2023.09.10 03:35:59 INFO  Started: Metals version 1.0.1 in folders '/Users/shota/Code/iohk/atala-prism-building-blocks' for client Visual Studio Code 1.82.0.
Sep 10, 2023 3:35:59 AM org.flywaydb.core.internal.license.VersionPrinter printVersionOnly
INFO: Flyway Community Edition 9.21.2 by Redgate
Sep 10, 2023 3:35:59 AM org.flywaydb.core.internal.license.VersionPrinter printVersion
INFO: See release notes here: https://rd.gt/416ObMi
Sep 10, 2023 3:35:59 AM org.flywaydb.core.internal.license.VersionPrinter printVersion
INFO: 
Sep 10, 2023 3:36:00 AM org.flywaydb.core.FlywayExecutor execute
INFO: Database: jdbc:h2:file:/Users/shota/Code/iohk/atala-prism-building-blocks/.metals/metals (H2 2.1)
Sep 10, 2023 3:36:00 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 4 migrations (execution time 00:00.028s)
Sep 10, 2023 3:36:00 AM org.flywaydb.core.internal.command.DbMigrate migrateGroup
INFO: Current version of schema "PUBLIC": 4
Sep 10, 2023 3:36:00 AM org.flywaydb.core.internal.command.DbMigrate logSummary
INFO: Schema "PUBLIC" is up to date. No migration necessary.
2023.09.10 03:36:00 INFO  time: initialize in 1.14s
2023.09.10 03:36:05 INFO  Attempting to connect to the build server...
2023.09.10 03:36:05 INFO  Bloop uses /Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home defined at /Users/shota/.bloop/bloop.json
2023.09.10 03:36:05 INFO  skipping build import with status 'Failed'
2023.09.10 03:36:06 INFO  tracing is disabled for protocol BSP, to enable tracing of incoming and outgoing JSON messages create an empty file at /Users/shota/Code/iohk/atala-prism-building-blocks/.metals/bsp.trace.json or /Users/shota/Library/Caches/org.scalameta.metals/bsp.trace.json
2023.09.10 03:36:06 INFO  Attempting to connect to the build server...
2023.09.10 03:36:06 INFO  Bloop uses /Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home defined at /Users/shota/.bloop/bloop.json
2023.09.10 03:36:06 INFO  Attempting to connect to the build server...
2023.09.10 03:36:06 INFO  Bloop uses /Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home defined at /Users/shota/.bloop/bloop.json
2023.09.10 03:36:06 INFO  tracing is disabled for protocol BSP, to enable tracing of incoming and outgoing JSON messages create an empty file at /Users/shota/Code/iohk/atala-prism-building-blocks/project/project/.metals/bsp.trace.json or /Users/shota/Library/Caches/org.scalameta.metals/bsp.trace.json
2023.09.10 03:36:06 INFO  tracing is disabled for protocol BSP, to enable tracing of incoming and outgoing JSON messages create an empty file at /Users/shota/Code/iohk/atala-prism-building-blocks/project/.metals/bsp.trace.json or /Users/shota/Library/Caches/org.scalameta.metals/bsp.trace.json
2023.09.10 03:36:06 INFO  time: Connected to build server in 1.28s
2023.09.10 03:36:06 INFO  Connected to Build server: Bloop v1.5.8
2023.09.10 03:36:07 INFO  time: Imported build in 0.56s
2023.09.10 03:36:07 INFO  time: reset stuff in 1ms
2023.09.10 03:36:07 INFO  time: updated main build targets in 0.11s
2023.09.10 03:36:07 INFO  time: updated ammonite build targets in 229μs
2023.09.10 03:36:07 INFO  time: updated scala-cli build targets in 311μs
2023.09.10 03:36:09 INFO  time: post update build targets stuff in 1.88s
2023.09.10 03:36:09 INFO  time: started file watcher in 0.83s
2023.09.10 03:36:12 INFO  running '/Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home/bin/java -Djline.terminal=jline.UnsupportedTerminal -Dsbt.log.noformat=true -Dfile.encoding=UTF-8 -jar /var/folders/r_/hmctw45x68763vr97hqgzcl00000gn/T/metals5129489566160781611/sbt-launch.jar -Dbloop.export-jar-classifiers=sources bloopInstall'
2023.09.10 03:36:15 INFO  [info] welcome to sbt 1.8.0 (AdoptOpenJDK Java 11.0.11)
2023.09.10 03:36:15 INFO  time: indexed library classpath in 5.7s
2023.09.10 03:36:15 INFO  time: indexed workspace SemanticDBs in 25ms
2023.09.10 03:36:16 INFO  [info] loading settings for project global-plugins from plugins.sbt ...
2023.09.10 03:36:18 INFO  [info] loading global plugins from /Users/shota/.sbt/1.0/plugins
2023.09.10 03:36:20 INFO  [info] loading settings for project atala-prism-building-blocks-build-build-build from metals.sbt ...
2023.09.10 03:36:20 INFO  [info] loading project definition from /Users/shota/Code/iohk/atala-prism-building-blocks/project/project/project
2023.09.10 03:36:22 INFO  [info] loading settings for project atala-prism-building-blocks-build-build from metals.sbt ...
2023.09.10 03:36:22 INFO  [info] loading project definition from /Users/shota/Code/iohk/atala-prism-building-blocks/project/project
2023.09.10 03:36:25 INFO  time: indexed workspace main sources in 9.35s
2023.09.10 03:36:25 INFO  time: indexed workspace ammonite sources in 4ms
2023.09.10 03:36:25 INFO  time: indexed workspace scala-cli sources in 687μs
2023.09.10 03:36:26 INFO  [success] Generated .bloop/atala-prism-building-blocks-build-build.json
2023.09.10 03:36:26 INFO  time: indexed library sources in 1.58s
2023.09.10 03:36:26 INFO  time: indexed library sources in 12ms
2023.09.10 03:36:26 INFO  time: indexed library sources in 9ms
2023.09.10 03:36:26 INFO  time: indexed workspace in 19s
2023.09.10 03:36:26 INFO  [success] Total time: 4 s, completed Sep 10, 2023, 3:36:26 AM
# WARNING: Unable to get Instrumentation. Dynamic Attach failed. You may add this JAR as -javaagent manually, or supply -Djdk.attach.allowAttachSelf
2023.09.10 03:36:26 INFO  [info] loading settings for project atala-prism-building-blocks-build from metals.sbt,plugins.sbt,sbt-env.sbt ...
2023.09.10 03:36:26 INFO  [info] loading project definition from /Users/shota/Code/iohk/atala-prism-building-blocks/project
# WARNING: Unable to attach Serviceability Agent. You can try again with escalated privileges. Two options: a) use -Djol.tryWithSudo=true to try with sudo; b) echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
Error while accessing @Contended value: class org.openjdk.jol.vm.ContendedSupport cannot access interface jdk.internal.vm.annotation.Contended (in module java.base) because module java.base does not export jdk.internal.vm.annotation to unnamed module @7e472d

2023.09.10 03:36:28 INFO  [success] Generated .bloop/atala-prism-building-blocks-build.json
2023.09.10 03:36:28 INFO  [success] Total time: 2 s, completed Sep 10, 2023, 3:36:29 AM
2023.09.10 03:36:30 INFO  [info] loading settings for project root from build.sbt,version.sbt ...
2023.09.10 03:36:30 INFO  [info] resolving key references (30585 settings) ...
2023.09.10 03:36:31 INFO  memory: definition index using 56.2M (0 lines Scala)
2023.09.10 03:36:31 INFO  memory: references index using 13.9M (0 elements)
2023.09.10 03:36:31 INFO  memory: workspace symbol index using 16.2M (504 elements)
2023.09.10 03:36:33 INFO  [info] set current project to root (in build file:/Users/shota/Code/iohk/atala-prism-building-blocks/)
2023.09.10 03:36:33 INFO  [info] .env detected (fileName=.env). About to configure JVM System Environment with new map
2023.09.10 03:36:33 ERROR WARNING: An illegal reflective access operation has occurred
2023.09.10 03:36:33 ERROR WARNING: Illegal reflective access by au.com.onegeek.sbtdotenv.DirtyEnvironmentHack$ (file:/Users/shota/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/nl/gn0s1s/sbt-dotenv_2.12_1.0/3.0.0/sbt-dotenv-3.0.0.jar) to field java.lang.ProcessEnvironment.theEnvironment
2023.09.10 03:36:33 ERROR WARNING: Please consider reporting this to the maintainers of au.com.onegeek.sbtdotenv.DirtyEnvironmentHack$
2023.09.10 03:36:33 ERROR WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
2023.09.10 03:36:33 ERROR WARNING: All illegal access operations will be denied in a future release
2023.09.10 03:36:33 INFO  [info] Configured .env environment
2023.09.10 03:36:34 INFO  memory: build targets using 61.1M
2023.09.10 03:36:34 INFO  memory: classpath symbol index using 2.79M
2023.09.10 03:37:42 ERROR java.lang.OutOfMemoryError
2023.09.10 03:37:42 ERROR   at java.base/java.lang.AbstractStringBuilder.hugeCapacity(AbstractStringBuilder.java:214)
2023.09.10 03:37:42 ERROR   at java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:206)
2023.09.10 03:37:42 ERROR   at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:173)
2023.09.10 03:37:42 ERROR   at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538)
2023.09.10 03:37:42 ERROR   at java.base/java.lang.StringBuilder.append(StringBuilder.java:174)
2023.09.10 03:37:42 ERROR   at scala.collection.mutable.StringBuilder.append(StringBuilder.scala:203)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce$appender$1.apply(TraversableOnce.scala:419)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce$appender$1.apply(TraversableOnce.scala:410)
2023.09.10 03:37:42 ERROR   at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:335)
2023.09.10 03:37:42 ERROR   at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:1111)
2023.09.10 03:37:42 ERROR   at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:1111)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce.addString(TraversableOnce.scala:424)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce.addString$(TraversableOnce.scala:407)
2023.09.10 03:37:42 ERROR   at scala.collection.AbstractTraversable.addString(Traversable.scala:108)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce.mkString(TraversableOnce.scala:377)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce.mkString$(TraversableOnce.scala:376)
2023.09.10 03:37:42 ERROR   at scala.collection.AbstractTraversable.mkString(Traversable.scala:108)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce.mkString(TraversableOnce.scala:379)
2023.09.10 03:37:42 ERROR   at scala.collection.TraversableOnce.mkString$(TraversableOnce.scala:379)
2023.09.10 03:37:42 ERROR   at scala.collection.AbstractTraversable.mkString(Traversable.scala:108)
2023.09.10 03:37:42 ERROR   at bloop.integrations.sbt.BloopDefaults$.$anonfun$bloopInstall$15(SbtBloop.scala:1153)
2023.09.10 03:37:42 ERROR   at sbt.util.LoggerContext$LoggerContextImpl$Log.log(LoggerContext.scala:122)
2023.09.10 03:37:42 ERROR   at sbt.internal.util.ManagedLogger.log(ManagedLogger.scala:41)
2023.09.10 03:37:42 ERROR   at sbt.util.Logger.error(Logger.scala:28)
2023.09.10 03:37:42 ERROR   at bloop.integrations.sbt.BloopDefaults$.$anonfun$bloopInstall$14(SbtBloop.scala:1153)
2023.09.10 03:37:42 ERROR   at bloop.integrations.sbt.BloopDefaults$.$anonfun$bloopInstall$14$adapted(SbtBloop.scala:1151)
2023.09.10 03:37:42 ERROR   at scala.Function1.$anonfun$compose$1(Function1.scala:49)
2023.09.10 03:37:42 ERROR   at scala.Function1.$anonfun$compose$1(Function1.scala:49)
2023.09.10 03:37:42 ERROR   at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
2023.09.10 03:37:42 ERROR   at sbt.std.Transform$$anon$4.work(Transform.scala:68)
2023.09.10 03:37:42 ERROR   at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
2023.09.10 03:37:42 ERROR   at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
2023.09.10 03:37:42 ERROR   at sbt.Execute.work(Execute.scala:291)
2023.09.10 03:37:42 ERROR   at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
2023.09.10 03:37:42 ERROR   at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
2023.09.10 03:37:42 ERROR   at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
2023.09.10 03:37:42 ERROR   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2023.09.10 03:37:42 ERROR   at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
2023.09.10 03:37:42 ERROR   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2023.09.10 03:37:42 ERROR   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2023.09.10 03:37:42 ERROR   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2023.09.10 03:37:42 ERROR   at java.base/java.lang.Thread.run(Thread.java:829)
2023.09.10 03:37:42 ERROR [error] [launcher] error during sbt launcher: java.lang.OutOfMemoryError
2023.09.10 03:37:42 INFO  time: ran 'sbt bloopInstall' in 1m30s
2023.09.10 03:37:42 ERROR sbt command failed: /Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home/bin/java -Djline.terminal=jline.UnsupportedTerminal -Dsbt.log.noformat=true -Dfile.encoding=UTF-8 -jar /var/folders/r_/hmctw45x68763vr97hqgzcl00000gn/T/metals5129489566160781611/sbt-launch.jar -Dbloop.export-jar-classifiers=sources bloopInstall
2023.09.10 03:37:42 INFO  Disconnecting from Bloop session...
2023.09.10 03:37:42 INFO  Shut down connection with build server.
2023.09.10 03:37:42 INFO  Shut down connection with build server.
2023.09.10 03:37:42 INFO  Shut down connection with build server.
2023.09.10 03:37:42 INFO  Attempting to connect to the build server...
2023.09.10 03:37:42 INFO  Bloop uses /Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home defined at /Users/shota/.bloop/bloop.json
2023.09.10 03:37:42 INFO  tracing is disabled for protocol BSP, to enable tracing of incoming and outgoing JSON messages create an empty file at /Users/shota/Code/iohk/atala-prism-building-blocks/.metals/bsp.trace.json or /Users/shota/Library/Caches/org.scalameta.metals/bsp.trace.json
2023.09.10 03:37:42 INFO  Attempting to connect to the build server...
2023.09.10 03:37:42 INFO  Bloop uses /Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home defined at /Users/shota/.bloop/bloop.json
2023.09.10 03:37:42 INFO  Attempting to connect to the build server...
2023.09.10 03:37:42 INFO  Bloop uses /Users/shota/Library/Caches/Coursier/arc/https/github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.11%252B9/OpenJDK11U-jdk_x64_mac_hotspot_11.0.11_9.tar.gz/jdk-11.0.11+9/Contents/Home defined at /Users/shota/.bloop/bloop.json
2023.09.10 03:37:42 INFO  tracing is disabled for protocol BSP, to enable tracing of incoming and outgoing JSON messages create an empty file at /Users/shota/Code/iohk/atala-prism-building-blocks/project/project/.metals/bsp.trace.json or /Users/shota/Library/Caches/org.scalameta.metals/bsp.trace.json
2023.09.10 03:37:42 INFO  tracing is disabled for protocol BSP, to enable tracing of incoming and outgoing JSON messages create an empty file at /Users/shota/Code/iohk/atala-prism-building-blocks/project/.metals/bsp.trace.json or /Users/shota/Library/Caches/org.scalameta.metals/bsp.trace.json
2023.09.10 03:37:42 INFO  time: Connected to build server in 0.22s
2023.09.10 03:37:42 INFO  Connected to Build server: Bloop v1.5.8
2023.09.10 03:37:43 INFO  time: Imported build in 0.22s
2023.09.10 03:37:43 INFO  time: reset stuff in 308μs
2023.09.10 03:37:43 INFO  time: updated main build targets in 25ms
2023.09.10 03:37:43 INFO  time: updated ammonite build targets in 1ms
2023.09.10 03:37:43 INFO  time: updated scala-cli build targets in 80μs
2023.09.10 03:37:43 INFO  time: post update build targets stuff in 41ms
2023.09.10 03:37:43 INFO  time: started file watcher in 0.26s
2023.09.10 03:37:46 INFO  compiling polluxanoncreds (3 scala sources)
2023.09.10 03:37:46 INFO  compiling shared (22 scala sources)
Sep 10, 2023 3:37:46 AM org.eclipse.lsp4j.jsonrpc.RemoteEndpoint handleCancellation
WARNING: Unmatched cancel notification for request id 2
2023.09.10 03:37:47 INFO  time: compiled polluxAnoncreds in 1.56s
2023.09.10 03:37:47 INFO  time: compiled shared in 1.05s
2023.09.10 03:37:49 INFO  time: indexed library classpath in 6.08s
2023.09.10 03:37:49 INFO  time: indexed workspace SemanticDBs in 0.2s
2023.09.10 03:37:51 INFO  time: indexed workspace main sources in 1.61s
2023.09.10 03:37:51 INFO  time: indexed workspace ammonite sources in 3ms
2023.09.10 03:37:51 INFO  time: indexed workspace scala-cli sources in 2ms
2023.09.10 03:37:51 INFO  time: indexed library sources in 0.65s
2023.09.10 03:37:51 INFO  time: indexed library sources in 3ms
2023.09.10 03:37:51 INFO  time: indexed library sources in 1ms
2023.09.10 03:37:51 INFO  time: indexed workspace in 8.93s
2023.09.10 03:37:52 INFO  compiling polluxanoncreds (3 scala sources)
2023.09.10 03:37:55 INFO  memory: definition index using 147M (6,743 lines Scala)
2023.09.10 03:37:58 INFO  compiling shared (22 scala sources)
2023.09.10 03:37:58 INFO  time: compiled polluxAnoncreds in 6.1s
2023.09.10 03:37:58 INFO  time: compiled shared in 0.16s
2023.09.10 03:37:58 INFO  memory: references index using 150M (101 elements)
2023.09.10 03:37:58 INFO  memory: workspace symbol index using 43.0M (504 elements)
2023.09.10 03:38:00 INFO  memory: build targets using 99.2M
2023.09.10 03:38:00 INFO  memory: classpath symbol index using 2.79M

The repository where I'm trying to import the build is open-source, so I can also share that: https://github.com/input-output-hk/atala-prism-building-blocks

olatheander commented 1 year ago

I have discovered an additional property of the problem; by request of my client I had the Scala source on an eCryptfs partition. As an experiment I moved the code to a regular partition and worked with it there for a bit and I have not had any OOM issues while doing so. The behavior is much more stable.

I expected everything to be a bit slower on an encrypted file system but I'm not sure why it should affect memory consumption the way it did.

tgodzik commented 12 months ago

This might be related to some hashes, though hard to say which ones, but maybe we are not recognizing something as the same and reading into memory more than needed :thinking:

We had similar issue previously https://github.com/scalameta/metals/issues/4623

odersky commented 11 months ago

I am having the similar issues since about a month. When I edit Dotty intensively, I get OOMs about twice a day. The Output reads:

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "pool-4-thread-1"

This makes Metals much less fund to use than what we had previously.

Have you tried to run metals with a memory profiler?

mbovel commented 10 months ago

I also got my CPU burning several times because of Metals:

➜  ~ ps aux         
USER               PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
mbovel           28304 860.9  7.5 36816248 1261084   ??  R     2:18PM 3413:10.43 /Users/mbovel/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.6%252B10/OpenJDK17U-jdk_x64_mac_hotspot_17.0.6_10.tar.gz/jdk-17.0.6+10/Contents/Home/bin/java -Xss4m -Xms100m -Dmetals.client=vscode -Xmx1G -classpath /Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/metals_2.13/1.1.0/metals_2.13-1.1.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.12/scala-library-2.13.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/mtags_2.13.12/1.1.0/mtags_2.13.12-1.1.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/mtags-java_2.13/1.1.0/mtags-java_2.13-1.1.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/guava/guava/32.1.3-jre/guava-32.1.3-jre.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/geirsson/metaconfig-core_2.13/0.12.0/metaconfig-core_2.13-0.12.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/openjdk/jol/jol-core/0.17/jol-core-0.17.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/swoval/file-tree-views/2.1.12/file-tree-views-2.1.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/undertow/undertow-core/2.2.20.Final/undertow-core-2.2.20.Final.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jboss/xnio/xnio-nio/3.8.11.Final/xnio-nio-3.8.11.Final.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/flywaydb/flyway-core/9.22.3/flyway-core-9.22.3.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/h2database/h2/2.1.214/h2-2.1.214.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/ipcsocket/ipcsocket/1.6.2/ipcsocket-1.6.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/epfl/scala/bsp4j/2.1.0-M7/bsp4j-2.1.0-M7.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/epfl/scala/bloop-launcher-core_2.13/1.5.11/bloop-launcher-core_2.13-1.5.11.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/lsp4j/org.eclipse.lsp4j/0.20.1/org.eclipse.lsp4j-0.20.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/lsp4j/org.eclipse.lsp4j.debug/0.20.1/org.eclipse.lsp4j.debug-0.20.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/dev/dirs/directories/26/directories-26.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/jdt/org.eclipse.jdt.core/3.25.0/org.eclipse.jdt.core-3.25.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.ant.core/3.5.500/org.eclipse.ant.core-3.5.500.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.compare.core/3.6.600/org.eclipse.compare.core-3.6.600.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.commands/3.9.500/org.eclipse.core.commands-3.9.500.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.contenttype/3.7.500/org.eclipse.core.contenttype-3.7.500.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.expressions/3.6.500/org.eclipse.core.expressions-3.6.500.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.filesystem/1.7.500/org.eclipse.core.filesystem-1.7.500.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.jobs/3.10.500/org.eclipse.core.jobs-3.10.500.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.resources/3.13.500/org.eclipse.core.resources-3.13.500.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.runtime/3.16.0/org.eclipse.core.runtime-3.16.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.core.variables/3.4.600/org.eclipse.core.variables-3.4.600.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.equinox.app/1.4.300/org.eclipse.equinox.app-1.4.300.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.equinox.common/3.10.600/org.eclipse.equinox.common-3.10.600.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.equinox.preferences/3.7.600/org.eclipse.equinox.preferences-3.7.600.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.equinox.registry/3.8.600/org.eclipse.equinox.registry-3.8.600.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.osgi/3.15.0/org.eclipse.osgi-3.15.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.team.core/3.8.700/org.eclipse.team.core-3.8.700.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.text/3.9.0/org.eclipse.text-3.9.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/mdoc-interfaces/2.3.7/mdoc-interfaces-2.3.7.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-dynamic_2.13/3.7.14/scalafmt-dynamic_2.13-3.7.14.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/epfl/scala/scalafix-interfaces/0.11.1/scalafix-interfaces-0.11.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/interface/1.0.19/interface-1.0.19.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/versions_2.13/0.3.2/versions_2.13-0.3.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/outr/scribe_2.13/3.12.2/scribe_2.13-3.12.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/outr/scribe-file_2.13/3.12.2/scribe-file_2.13-3.12.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/outr/scribe-slf4j_2.13/3.12.2/scribe-slf4j_2.13-3.12.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ujson_2.13/3.1.3/ujson_2.13-3.1.3.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/requests_2.13/0.8.0/requests_2.13-0.8.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalameta_2.13/4.8.12/scalameta_2.13-4.8.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/semanticdb-scalac-core_2.13.12/4.8.12/semanticdb-scalac-core_2.13.12-4.8.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/ammonite/ammonite-runner_2.13/0.4.0/ammonite-runner_2.13-0.4.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.13/2.2.0/scala-xml_2.13-2.2.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-parallel-collections_2.13/1.0.4/scala-parallel-collections_2.13-1.0.4.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/virtuslab/scala-cli/scala-cli-bsp/1.0.4/scala-cli-bsp-1.0.4.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/mtags-shared_2.13.12/1.1.0/mtags-shared_2.13.12-1.1.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/geny_2.13/1.0.0/geny_2.13-1.0.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-java8-compat_2.13/1.0.2/scala-java8-compat_2.13-1.0.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jsoup/jsoup/1.16.1/jsoup-1.16.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/lz4/lz4-java/1.8.0/lz4-java-1.8.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/googlecode/java-diff-utils/diffutils/1.3.0/diffutils-1.3.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/mtags-interfaces/1.1.0/mtags-interfaces-1.1.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/checkerframework/checker-qual/3.37.0/checker-qual-3.37.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.21.1/error_prone_annotations-2.21.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/j2objc/j2objc-annotations/2.8/j2objc-annotations-2.8.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/geirsson/metaconfig-pprint_2.13/0.12.0/metaconfig-pprint_2.13-0.12.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/paiges-core_2.13/0.4.3/paiges-core_2.13-0.4.3.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-collection-compat_2.13/2.11.0/scala-collection-compat_2.13-2.11.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.12/scala-reflect-2.13.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jboss/logging/jboss-logging/3.4.1.Final/jboss-logging-3.4.1.Final.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jboss/xnio/xnio-api/3.8.11.Final/xnio-api-3.8.11.Final.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jboss/threads/jboss-threads/3.1.0.Final/jboss-threads-3.1.0.Final.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-toml/2.15.2/jackson-dataformat-toml-2.15.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/code/gson/gson/2.10.1/gson-2.10.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.13.0/jna-5.13.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna-platform/5.13.0/jna-platform-5.13.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/lsp4j/org.eclipse.lsp4j.generator/0.20.1/org.eclipse.lsp4j.generator-0.20.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/lsp4j/org.eclipse.lsp4j.jsonrpc/0.20.1/org.eclipse.lsp4j.jsonrpc-0.20.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/epfl/scala/sockets/1.5.11/sockets-1.5.11.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/epfl/scala/bloopgun-core_2.13/1.5.11/bloopgun-core_2.13-1.5.11.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/lsp4j/org.eclipse.lsp4j.jsonrpc.debug/0.20.1/org.eclipse.lsp4j.jsonrpc.debug-0.20.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-interfaces/3.7.14/scalafmt-interfaces-3.7.14.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/typesafe/config/1.4.2/config-1.4.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/2.0.7/slf4j-api-2.0.7.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/outr/perfolation_2.13/1.2.9/perfolation_2.13-1.2.9.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/sourcecode_2.13/0.3.1/sourcecode_2.13-0.3.1.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/outr/moduload_2.13/1.1.7/moduload_2.13-1.1.7.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-core_2.13/3.1.3/upickle-core_2.13-3.1.3.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/parsers_2.13/4.8.12/parsers_2.13-4.8.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scalap/2.13.12/scalap-2.13.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.12/scala-compiler-2.13.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-launcher_2.13/2.0.16/coursier-launcher_2.13-2.0.16.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.24.4/protobuf-java-3.24.4.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fansi_2.13/0.4.0/fansi_2.13-0.4.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/wildfly/common/wildfly-common/1.5.4.Final/wildfly-common-1.5.4.Final.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/wildfly/client/wildfly-client-config/1.0.1.Final/wildfly-client-config-1.0.1.Final.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.15.2/jackson-databind-2.15.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.15.2/jackson-core-2.15.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/xtend/org.eclipse.xtend.lib/2.28.0/org.eclipse.xtend.lib-2.28.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/me/vican/jorge/snailgun-cli_2.13/0.4.0/snailgun-cli_2.13-0.4.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/zeroturnaround/zt-exec/1.12/zt-exec-1.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-nop/2.0.7/slf4j-nop-2.0.7.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/interface-svm-subs/1.0.18/interface-svm-subs-1.0.18.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/github/plokhotnyuk/jsoniter-scala/jsoniter-scala-core_2.13/2.13.3.2/jsoniter-scala-core_2.13-2.13.3.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/trees_2.13/4.8.12/trees_2.13-4.8.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/java-diff-utils/java-diff-utils/4.12/java-diff-utils-4.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline/3.22.0/jline-3.22.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.15.2/jackson-annotations-2.15.2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/xtext/org.eclipse.xtext.xbase.lib/2.28.0/org.eclipse.xtext.xbase.lib-2.28.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/eclipse/xtend/org.eclipse.xtend.lib.macro/2.28.0/org.eclipse.xtend.lib.macro-2.28.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/me/vican/jorge/snailgun-core_2.13/0.4.0/snailgun-core_2.13-0.4.0.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/github/scopt/scopt_2.13/4.0.0-RC2/scopt_2.13-4.0.0-RC2.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/common_2.13/4.8.12/common_2.13-4.8.12.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/thesamet/scalapb/scalapb-runtime_2.13/0.11.13/scalapb-runtime_2.13-0.11.13.jar:/Users/mbovel/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/thesamet/scalapb/lenses_2.13/0.11.13/lenses_2.13-0.11.13.jar scala.meta.metals.Main

This process is a daemon:

➜  ~ ps aux -A -o ppid | head
USER               PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND           PPID
mbovel           28304 949.4  7.5 36816248 1260972   ??  R     2:18PM 3447:28.56 /Users/mbovel/Li     1

So even after exiting VSCode, it keeps running. Is this expected?

Further inspecting the process with VisualVM showed that the heap memory was full (1GB/1GB). So I searched for "memory leak" in this repo and found this issue.

My current "workaround" is to kill all Java processes when this happens:

jps -q | xargs kill -9

Note: I am on macOS 13.2.1, with Metals 1.1.0 and Java 17.0.6 from Eclipse Adoptium.

Note 2: I had 3 Scala projects opened when this happened with the following info. One with "Build definition is coming from scala-cli. Build server currently being used is scala-cli v1.0.4" and two with "Build definition is coming from sbt. Build server currently being used is Bloop v1.5.11."

odersky commented 10 months ago

My issue was that I had several subdirectories that were inadvertently their own projects created by scala-cli runner. I think there was a scala.json file for each of them (not 100% sure about the name). The result was that several compilers were competing compiling the same files. Once I deleted the project files, everything was OK again.

It would be good to devise a scheme how that issue could be prevented. Maybe Metals could scan for embedded scala.json files and offer to delete them?

tgodzik commented 10 months ago

@mbovel look like your case is where the process hung, it might be a memory leak or the project might be large enough to require more memory. Anyway, it should close when VS Code itself closes, we need to check why that is not the case.

tgodzik commented 10 months ago

Looking a bit into the API it seems it's all done behind the scenes. We only provide the command to invoke and later we do .stop() which if it doesn't work we have no way of seeing. It looks there was in issue around that and it should be resolved, but I guess it wasn't :thinking:

https://github.com/microsoft/vscode-languageserver-node/issues/834