search

fsharp / fsharp-compiler-docs

Doc build for FSharp.Compiler.Service
https://fsharp.github.io/fsharp-compiler-docs
MIT License
279 stars 123 forks source link

Incremental builds cache does not work well for large solutions #121

Closed vasily-kirichenko closed 10 years ago

vasily-kirichenko commented 10 years ago

In certain scenarios ("Find all references" and solution-wide "Rename refactoring") we need to check all (or a large subset of) F# projects in a solution. For large solutions, which can contains several dozens of F# projects, MRU cache of default size 3 works really bad: subsequent parsing / type checking are as slow as the very first one.

I managed to make the cache works well for a solution containing ~40 small F# projects by increasing cache size to 150 here https://github.com/fsharp/FSharp.Compiler.Service/blob/master/src/fsharp/vs/service.fs#L54 (and I changed the 75 to 150 here https://github.com/fsharp/FSharp.Compiler.Service/blob/master/src/fsharp/InternalCollections.fs#L28). VS process took ~1.4GB RAM which is OK.

The discussion started in VFPT here https://github.com/fsprojects/VisualFSharpPowerTools/pull/350

So, I think (as @dsyme also suggested) we need a different cache strategy for such scenarios (or even just make the cache infinite).

This is a rather severe bug (at least for me) as it actually blocks using "Find all references" on real-world solutions.

dungpa commented 10 years ago

Is there any chance to provide alternative cache mechanisms such as LRU, LFU, etc. and let clients choose which mechanism suit their purposes?

quasilord commented 10 years ago

Initially we will add an optional project cache size parameter to InteractiveChecker. This is the number of strong "project builder" references in the project builder cache. Beware that setting the figure too high can result in out-of-memory.

If you would like a different cache strategy could you consider sending a pull request?

dungpa commented 10 years ago

Thanks. A cache size parameter would be good for now. We understand the implication and will use it sensibly.

quasilord commented 10 years ago

FCS 0.0.45 now has the optional parameter for the strong project references in the project builder cache. Please make sure the user can set this value in the Visual F# Power Tools options in case they start hitting out-of-memory problems?

Closing this for now as that should be enough to fix the basic problem.