search

rust-lang / rust-analyzer

A Rust compiler front-end for IDEs
https://rust-analyzer.github.io/
Apache License 2.0
13.77k stars 1.52k forks source link

Load files into the VFS in parallel on startup #17373

Open Veykril opened 2 weeks ago

Veykril commented 2 weeks ago

We currently load them from a single thread which can make startup very slow for bigger projects as we re-query the project layout over and over again until all files are loaded (this is a separate issue that we should fix, that is wait with that until all files are loaded into memory).

davidbarsky commented 2 weeks ago

For some additional context from Zulip, the current single-threaded loading behavior is especially problematic on EdenFS, which really wants tooling to send as many IO requests as possible in parallel.

davidbarsky commented 2 weeks ago

this is a separate issue that we should fix, that is wait with that until all files are loaded into memory.

I did some rough, back-of-napkin benchmarking on this: rust-analyzer spends 5 seconds waiting on rustc for every 100 crates it loads (just printed Instant::elapsed::as_millis...). Given that total crate dependency graphs of over 1000 are not uncommon for larger Rust projects in the wild, it's likely that people are unnecessarily waiting for over a minute.

What makes this fix a little more complicated is that during project loading, there isn't a clear end-state: rust-analyzer just sorta repeatedly calls GlobalState::fetch_workspaces.