rust-lang / rust-analyzer

A Rust compiler front-end for IDEs
https://rust-analyzer.github.io/
Apache License 2.0
14.21k stars 1.6k forks source link

Terrible memory usage on `Rikorose/DeepFilterNet` #16817

Open lnicola opened 7 months ago

lnicola commented 7 months ago

https://github.com/Rikorose/DeepFilterNet

I stopped it at 24 GB, might grow more.

Per-query memory usage:
   792mb      1 LocalRootsQuery
   510mb     68 ValueTyQuery
   383mb      2 MirBodyQuery
   282mb      0 LibrarySymbolsQuery
   186mb   8462 TyQuery
   147mb  39513 MacroArgQuery
   126mb  42327 FileItemTreeQuery
   109mb   3629 ParseQuery
    81mb   6370 FileTextQuery

Also has interesting tidbits like:

   55066ms handle_code_action
      8125ms diagnostics
     43224ms find_path (251 calls)
       43221ms calculate_best_path (117 calls)
         43214ms find_path_for_module (630 calls)
           43212ms calculate_best_path (622 calls)
             43177ms find_path_for_module (6852 calls)
               43159ms calculate_best_path (5195 calls)
                 42888ms find_path_for_module (51238 calls)
                   42757ms calculate_best_path (35025 calls)
                     41041ms find_path_for_module (159125 calls)
                       40660ms calculate_best_path (88283 calls)
                         36232ms find_path_for_module (589883 calls)
                           34812ms calculate_best_path (345574 calls)
                             11414ms DefDatabase::import_map @ krate = Idx::<CrateData>(0)  (13477386 calls)
                             18188ms find_path_for_module (1061990 calls)
                               16103ms calculate_best_path (273152 calls)
                                  8983ms DefDatabase::import_map @ krate = Idx::<CrateData>(0)  (10652928 calls)

Which does seem a bit excessive?

Veykril commented 7 months ago

792mb 1 LocalRootsQuery

How? 😓

lnicola commented 7 months ago

According to perf, it's spending a lot of time in the tracing/profiling code:

image

lnicola commented 7 months ago

How? 😓

No idea, maybe bad measurement?

I didn't change anything and now I can't reproduce this any more :confused:.

lnicola commented 7 months ago

And for some reason we can't find clap in enhance_wav.rs:

image

Veykril commented 7 months ago

According to perf, it's spending a lot of time in the tracing/profiling code:

image

On that note, we compile with all tracing levels statically enabled don't we. We might want to raise the minimum to info for that (unless thats the default in release, I don't recall)