search

scalameta / metals

Scala language server with rich IDE features 🚀
https://scalameta.org/metals/
Apache License 2.0
2.04k stars 318 forks source link

Indexing never completes on large Scala codebase #6552

Open tmiseikis opened 2 days ago

tmiseikis commented 2 days ago

Describe the bug

Hi, we have a relatively large Scala codebase in terms of lines of code, but fairly average in terms of number of files. It's a bit of a special codebase, because the code was not hand written, but rather transpiled from a legacy domain specific language.

➜ tokei -t scala
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Scala                5553      1860592      1323854       149163       387575
===============================================================================

As you can see the project has over 1.3 million lines of Scala code across 5553 files.

When we open this project in Metals, we can see that the "indexing" phase never finishes. I have the project opened right now and the indexing has been running for nearly 2 hours:

image

We have witnessed the indexing phase running for over 3 hours before and never finishing.

We do not really see anything interesting in the metals.log besides warnings like this when trying to navigate to a definition:

2024.06.28 11:01:54 WARN  Could not load snapshot text for /Users/user/projects/xxx/SomeFile.scala
2024.06.28 11:01:54 WARN  Could not load snapshot text for /Users/user/projects/xxx/SomeFile.scala
2024.06.28 11:01:54 WARN  Using indexes to guess the definition of getDescription

Are there any known problems/limitations in Metals when trying to index large Scala files? Do you know if we could enable some additional logging to see what Metals is actually doing during the indexing phase? Do you perhaps have any other suggestions that we could investigate or try out?

Thanks.

Expected behavior

Indexing phase completes in a reasonable amount of time.

Operating system

macOS

Editor/Extension

Nvim (nvim-metals)

Version of Metals

v1.3.2

Extra context or search terms

No response

tgodzik commented 2 days ago

Thanks for reporting! We should not have any issues indexing a large workspace, but it might have never been tested on a one similar to yours. IT would be worth takin a look if it's actually not hanging due to GC issues. Might be worth bumping -Xmx option in metals.serverProeprties. You could also take a look at Visualvm or similar to monitor if there is anything actually being calculated.

You can also add -Dmetals.loglevel=debug and maybe -Dmetals.statistics=all

How large are specific files? We could try to generate something similar to take a look