instead of "blocking" on .collect()'ing all enumerated project files (via ignore) into a Vec and then proceeding into the (parallelized) query-matching phase, parallelize the enumerating of project files and start "flowing" those through into query-matching as they are enumerated
I did this because it was seeming like there was a noticeable initial "pause" before any results started showing up and I assumed that was the "enumerating project files" phase, which made me think that it would probably be a significant improvement to overall runtime to not block on that. Indeed here is a basic timing graph from before this branch:
And after:
In terms of roughly-measured (via time) overall runtime, against the Typescript compiler in Rust codebase it looks like it went from ~460-480ms to ~280-300ms so maybe a 35-40% speedup?
helixbass
commented
1 year ago
In this PR:
.collect()'ing all enumerated project files (viaignore) into aVecand then proceeding into the (parallelized) query-matching phase, parallelize the enumerating of project files and start "flowing" those through into query-matching as they are enumeratedI did this because it was seeming like there was a noticeable initial "pause" before any results started showing up and I assumed that was the "enumerating project files" phase, which made me think that it would probably be a significant improvement to overall runtime to not block on that. Indeed here is a basic timing graph from before this branch:
And after:
In terms of roughly-measured (via
time) overall runtime, against the Typescript compiler in Rust codebase it looks like it went from ~460-480ms to ~280-300ms so maybe a 35-40% speedup?Based on
filter-plugin-c-ffi