search

rust-lang / cargo-bisect-rustc

Bisects rustc, either nightlies or CI artifacts
https://rust-lang.github.io/cargo-bisect-rustc/
Apache License 2.0
177 stars 55 forks source link

Run downloads in parallel/asynchronously #120

Open camelid opened 3 years ago

camelid commented 3 years ago

I think this would make bisecting faster.

The only downside I see is that it would be more difficult to have progress bars for each of the components. Perhaps we could have one progress bar that shows the total number of bytes that have been/need to be downloaded? Then each thread/async fn would lock the progress bar and update it every 1 MB or so.

Mark-Simulacrum commented 3 years ago

I've always been of the opinion that this shouldn't make downloads faster because we should be entirely limited by network bandwidth -- that said, it would be good to confirm this (e.g., with curl or something running in parallel).

camelid commented 3 years ago

Yeah, unfortunately my internet ends up being 4 MB/s download speed, so bisecting can take a while :/

I use --preserve, so that helps, but then I use up a ton of disk space.

Mark-Simulacrum commented 3 years ago

What we can do is parallelize compilation and downloading the next batch of artifacts. It would mean branch predicting that -- or downloading both -- but that seems not unreasonable. I guess if the compilation is fast enough it's not actually a win though.

bjorn3 commented 3 years ago

Re branch predicting: I once read that it can sometimes be quicker to divide the range in three using two comparisons and branches rather than dividing the range in two. The former biases the first branch to not taken instead of the pathological case of taken and not taken being equally likely.