`dart:analysis_server` takes 19GB memory

[fzyzcjy]

Well, I have reported here https://github.com/dart-lang/sdk/issues/41793#issuecomment-1554378872, but I am not sure whether it is more reasonable to create a new issue, since the memory size is really really a lot IMHO. The original issue complains when it takes <2GB, while here it is nearly 20GB!

[srawlins]

@fzyzcjy I'm going to close this issue as a few folks on the Dart team have landed many memory improvements in the Analyzer code bases over the last year, and even over the last six months (see below). I won't promise that your memory problems are solved, but I think at this point, it would be most effective to sort of rebase the discussion.

Anyone else experiencing similar "giant memory usage" issues with the Dart Analyzer, and you'd like to document a bit your situation, also please follow these instructions:

First, be sure to bump to the latest Dart, 3.2.0, as it has some performance improvements over 3.1.0. Then, if you experience what you feel is an inordinate amount of memory being used (we don't have any SLOs yet in this space, but anything north of 5GB is probably noteworthy):

1. We have a public document with instructions how to setup auto-snapshotting for analyzer in case of huge memory usage: https://docs.google.com/document/d/1mxCKhvyHUtGVhIH5Apx9y7V7itqCxAQPse6ghbjaDMM
2. You can file an issue in this repo and send us some other basic info:
   • How many .dart files are under the analysis root for the IDE?
   • In the Diagnostics pages, you may want to look at Memory, Plugins, how many Analysis Contexts exist, etc.
   • How long has the process been running? If you restart your IDE or and/or DAS, does it use less memory? What about hours later? What about days later?

Recent memory-reducing changes:

Dart ~ 2.19.0, Jan 2023

• https://github.com/dart-lang/sdk/commit/9306752ba71352bd8ac39594ad151c34c8ab128a - large memory reduction
• https://github.com/dart-lang/sdk/commit/b785f7dba85e03d5307f99fb1ee5ec8286d39ece - small memory reduction
• https://github.com/dart-lang/sdk/commit/91c88ab06710b4e6ec6bec4b658a46e9c37b9b61 - small memory reduction
• https://github.com/dart-lang/sdk/commit/4193bb38468a109d8df7e62940f8949695a86d07 - small memory reduction

Dart ~3.0.0, May 2023

• https://github.com/dart-lang/sdk/commit/3378c55f7d8c21b09a145e5125bac34309f4e431 - small memory reduction

Dart ~3.1.0, August 2023

• https://github.com/dart-lang/sdk/commit/bda10d798700e6178f58d553160f8201674f06dd - large memory reduction
• https://github.com/dart-lang/sdk/commit/5e08794bbfc2c57b5b0522c6e2b22b419f6aa115 - memory leak fix
• https://github.com/dart-lang/sdk/commit/5aee0f61b6301099bb25a36d2b2ff197541d3c5b - memory leak fix (also cherry-picked into Dart 3.0.6)
• https://github.com/dart-lang/sdk/commit/3dd9389c1ff84fd48ba8cb69536c9bdfc07ea20f - memory leak fix (also cherry-picked into Dart 3.0.6)

Dart ~3.2.0, November 2023

• https://github.com/dart-lang/sdk/commit/aba5663d0a1f0dfcb94a78e49f577d19fca5653e - large memory reduction
• https://github.com/dart-lang/sdk/commit/ef2422ab892f96af381c4d22747e3e02d4a4ff8b - large memory reduction
• https://github.com/dart-lang/sdk/commit/f28d4a5e1d8899b7344c1c79e2eb9017a70f1677 - small memory reduction
• https://github.com/dart-lang/sdk/commit/2bbf99351e0274643b8ab0c14904efbb650ded8b - large memory reduction
• https://github.com/dart-lang/sdk/commit/807d1cf47be10c4ba73d8bc348ac291e5b046b12 - large memory reduction
• https://github.com/dart-lang/sdk/commit/5f23e5b02d1a80afd138f27a0b12b6517d481f3e - large memory reduction

(There were also speed-up improvements made in this time frame, and improvements to our analytics, logging, benchmarking, and platform coverage, but I'm not documenting those here on this memory-focused issue.)

[fzyzcjy]

Thank you and the Dart team for improving this! I will try Dart 3.2 later and report if anything goes wrong.

[RichardGreat]

I faced the same problem in flutter 3.7.12 but upgrading flutter version was not a option because I have a huge code base with 3-4 big modules which I have to migrate all my modules to dart null safety, I found the solution which worked on me the problem was in example project of an each module, which created I cyclic dependency source analyzing : module -> example -> dependency -> module -> example ... etc. and I reviewed my module examples and remove module dependency from pubspec.yaml file and now it helped and now it takes at most 1GB in ram

[polina-c]

For those who still experience this, you can enable auto memory snapshotting for analyzer.

Do not worry, it will not fill all you hard drive with snapshots, it will stop at the configured limit.

Here is the steps how to configure your IDE to turn on analyzer autosnapshotting: https://docs.google.com/document/d/1mxCKhvyHUtGVhIH5Apx9y7V7itqCxAQPse6ghbjaDMM/edit

Then, with DevTools starting flutter 3.22.0-1.0.pre.10 or fresher (you do not have to migrate your project, just switch to flutter master to get DevTools) you can import and diff the collected snapshots. Or, if the snapshots do not contain sensitive information, just share them here so that analyzer team can check them.