Very slow checking of installed crate versions on `cargo binstall`

[0atman]

If you, like me, maintain a large list of system tools in Rust and re-run the install line now and then to keep up to date, you'll notice a very long delay checking each crate for being up to date, even if no version bump has happened.

After package indexing, which is very fast, this process is entirely local and so shouldn't be slow. Yet it is, about 1s delayed per item.

Here's my preposterous install line that I will take no notes on:

$ time ~/.cargo/bin/cargo binstall -y --locked cargo-info ripgrep ncspot du-dust exa bat irust nu starship zellij bacon cargo-watch porsmo speedtest-rs gitui wiki-tui evcxr_jupyter rtx-cli mprocs procs joshuto rsdate xh fd-find sd alacritty paru gitoxide bob-nvim chatgpt-cli turbocommit aichat cargo-modules gitoxide sqlx-cli dioxus-cli termscp bottom silicon cargo-nextest iamb tuifeed tealdeer just

 INFO resolve: Resolving package: 'aichat'
 INFO resolve: Resolving package: 'bacon'
 INFO resolve: Resolving package: 'cargo-watch'
 INFO resolve: Resolving package: 'alacritty'
 INFO resolve: Resolving package: 'cargo-modules'
 INFO resolve: Resolving package: 'cargo-nextest'
 INFO resolve: Resolving package: 'just'
 INFO resolve: Resolving package: 'chatgpt-cli'
 INFO resolve: Resolving package: 'mprocs'
 INFO resolve: Resolving package: 'fd-find'
 INFO resolve: Resolving package: 'paru'
 INFO resolve: Resolving package: 'gitoxide'
 INFO resolve: Resolving package: 'porsmo'
 INFO resolve: Resolving package: 'speedtest-rs'
 INFO resolve: Resolving package: 'gitui'
 INFO resolve: Resolving package: 'sqlx-cli'
 INFO resolve: Resolving package: 'termscp'
 INFO resolve: Resolving package: 'tealdeer'
 INFO resolve: Resolving package: 'tuifeed'
 INFO resolve: Resolving package: 'sd'
 INFO resolve: Resolving package: 'bottom'
 INFO resolve: Resolving package: 'ncspot'
 INFO resolve: Resolving package: 'silicon'
 INFO resolve: Resolving package: 'nu'
 INFO resolve: Resolving package: 'starship'
 INFO resolve: Resolving package: 'zellij'
 INFO resolve: Resolving package: 'procs'
 INFO resolve: Resolving package: 'iamb'
 INFO resolve: Resolving package: 'rsdate'
 INFO resolve: Resolving package: 'du-dust'
 INFO resolve: Resolving package: 'bob-nvim'
 INFO resolve: Resolving package: 'turbocommit'
 INFO resolve: Resolving package: 'evcxr_jupyter'
 INFO resolve: Resolving package: 'dioxus-cli'
 INFO resolve: Resolving package: 'bat'
 INFO resolve: Resolving package: 'xh'
 INFO resolve: Resolving package: 'cargo-info'
 INFO resolve: Resolving package: 'irust'
 INFO resolve: Resolving package: 'rtx-cli'
 INFO resolve: Resolving package: 'ripgrep'
 INFO resolve: Resolving package: 'joshuto'
 INFO resolve: Resolving package: 'wiki-tui'
 INFO resolve: Resolving package: 'exa'

Output immediately jumps to here, then spins 1s for each of the following lines:

 INFO resolve: aichat v0.8.0 is already installed, use --force to override
 INFO resolve: bacon v2.12.1 is already installed, use --force to override
 INFO resolve: cargo-watch v8.4.0 is already installed, use --force to override
 INFO resolve: cargo-modules v0.9.2 is already installed, use --force to override
 INFO resolve: alacritty v0.12.2 is already installed, use --force to override
 INFO resolve: cargo-nextest v0.9.57 is already installed, use --force to override
 INFO resolve: just v1.14.0 is already installed, use --force to override
 INFO resolve: chatgpt-cli v0.3.1 is already installed, use --force to override
 INFO resolve: mprocs v0.6.4 is already installed, use --force to override
 INFO resolve: fd-find v8.7.0 is already installed, use --force to override
 INFO resolve: paru v1.11.0 is already installed, use --force to override
 INFO resolve: porsmo v0.2.2 is already installed, use --force to override
 INFO resolve: gitoxide v0.28.0 is already installed, use --force to override
 INFO resolve: speedtest-rs v0.1.4 is already installed, use --force to override
 INFO resolve: gitui v0.23.0 is already installed, use --force to override
 INFO resolve: sqlx-cli v0.7.1 is already installed, use --force to override
 INFO resolve: termscp v0.12.1 is already installed, use --force to override
 INFO resolve: tealdeer v1.6.1 is already installed, use --force to override
 INFO resolve: tuifeed v0.3.2 is already installed, use --force to override
 INFO resolve: sd v0.7.6 is already installed, use --force to override
 INFO resolve: ncspot v0.13.4 is already installed, use --force to override
 INFO resolve: bottom v0.9.4 is already installed, use --force to override
 INFO resolve: silicon v0.5.1 is already installed, use --force to override
 INFO resolve: nu v0.83.1 is already installed, use --force to override
 INFO resolve: zellij v0.37.2 is already installed, use --force to override
 INFO resolve: starship v1.16.0 is already installed, use --force to override
 INFO resolve: procs v0.14.0 is already installed, use --force to override
 INFO resolve: iamb v0.0.8 is already installed, use --force to override
 INFO resolve: rsdate v0.5.0 is already installed, use --force to override
 INFO resolve: bob-nvim v2.4.1 is already installed, use --force to override
 INFO resolve: turbocommit v0.12.1 is already installed, use --force to override
 INFO resolve: du-dust v0.8.6 is already installed, use --force to override
 INFO resolve: evcxr_jupyter v0.15.1 is already installed, use --force to override
 INFO resolve: dioxus-cli v0.4.0 is already installed, use --force to override
 INFO resolve: xh v0.18.0 is already installed, use --force to override
 INFO resolve: bat v0.23.0 is already installed, use --force to override
 INFO resolve: cargo-info v0.7.6 is already installed, use --force to override
 INFO resolve: irust v1.70.0 is already installed, use --force to override
 INFO resolve: rtx-cli v2023.8.0 is already installed, use --force to override
 INFO resolve: ripgrep v13.0.0 is already installed, use --force to override
 INFO resolve: joshuto v0.9.5 is already installed, use --force to override
 INFO resolve: wiki-tui v0.8.2 is already installed, use --force to override
 INFO resolve: exa v0.10.1 is already installed, use --force to override
 INFO Done in 42.193393143s

real    0m42.236s
user    0m0.323s
sys 0m0.062s

Is there a sleep(1) in there somewhere? :sweat_smile:

[NobodyXu]

Is there a sleep(1) in there somewhere? 😅

Actually yes, you found out the cause faster than I do.

Our crates.io client has a one request per each second rate limit, I think it was copied from crates_io_api.

[NobodyXu]

We could remove that and just the global rate limit policy.

[NobodyXu]

@0atman We have released cargo-binstall v1.3.0 which should fixed this by switching to sparse index and use the global rate-limit instead. Please have a try and I'd like your feedback!