High memory usage for profiles with lots of tracks

[ishitatsuyuki]

The profiler seems to consume 838MB of memory with a profile that has 6000 tracks. I can't upload the profile as it's from a corp workstation, but I'll try to recreate one at home later (I believe anything that has the same amount of track should repro).

Chrome's sampling allocation profiler points to getCPUProcessedThreads as the likely culprit. This instantiates the selector graph for all threads in the profile, and the memory cost of creating the selector themselves seem to be dominating.

I found some low hanging fruits in reselect which cuts the memory usage down to 658MB. However, I doubt I can go much further than that.

For long term, I think we might want one of:

• Eagerly running the selector logic for getCPUProcessedThreads, without the memory overhead of creating all the selectors
• Share some of the selector logic across all parametizations, e.g. how dependencies can be computed. The dependency graph cannot be shared between parametizations, but they can be instantiated more lazily than now.

[julienw]

I believe it's the same root cause as https://github.com/firefox-devtools/profiler/issues/4815

I think @canova already looked at this but it wasn't that obvious in the end.

[ishitatsuyuki]

Just for the record, this is just loading the profile (with 15 top tracks displayed as default), not expanding all 6000 tracks.

4815 looks similar but I'm not 100% sure it's the same root cause.

[julienw]

Ah yeah, then I believe you're right, that's another issue. Thanks for looking at the possible solutions. We went from a situation with 4 content processes to many threads over many processes, and now 6k tracks. So I'm not surprised our current code doesn't scale at this level.