Dynamically install required package manager version at runtime

[JamieMagee]

Code improvement description

Currently, we define static versions of each programming language or package manager that Dependabot uses at runtime. For example, npm, rust, python, etc.

Unfortunately, this means that Dependabot can sometimes be a bottleneck for users who upgrade to a newer version that we don't currently install in our container images. For example see #9249.

We could remove this bottleneck, and source of user frustration, by dynamically installing the required version of a programming language at runtime if a specific version is detected in the repository. For .NET, that would be looking in global.json; for npm that would be looking at the engines property in package.json, etc.  

[yeikel]

What should happen if the property is not available? Fallback?

For npm for example, engines is optional

[JamieMagee]

Yeah, I think this should be in addition to the current default version that is installed in the container images.

[RobJellinghaus]

This seems like a per-ecosystem problem. Would it be more appropriate to have one issue per ecosystem?

On the Rust front in particular, Dependabot already does this without even knowing it :-D Specifically, cargo update when run by Dependabot will actually update the Rust toolchain, if the repository being updated has a rust-toolchain.toml file that specifies a particular version.

Here's some log output from a dependabot-cli run updating a test repository that specifies channel = "1.77" in its rust-toolchain.toml file:

updater | 2024/05/06 16:00:19 INFO Latest version is 1.0.82
updater | Setting CARGO_REGISTRIES_RUST_PUBLICPACKAGES_TOKEN to 'placeholder_token' because dependabot-cli proxy will override it anyway
updater | Setting CARGO_REGISTRIES_RUST_CRATEREVIEW_TOKEN to 'placeholder_token' because dependabot-cli proxy will override it anyway
  proxy | 2024/05/06 16:00:19 [024] GET https://static.rust-lang.org:443/dist/channel-rust-1.77.toml.sha256
  proxy | 2024/05/06 16:00:20 [024] 200 https://static.rust-lang.org:443/dist/channel-rust-1.77.toml.sha256
  proxy | 2024/05/06 16:00:20 [026] GET https://static.rust-lang.org:443/dist/channel-rust-1.77.toml
  proxy | 2024/05/06 16:00:20 [026] 200 https://static.rust-lang.org:443/dist/channel-rust-1.77.toml
  proxy | 2024/05/06 16:00:20 [028] GET https://static.rust-lang.org:443/dist/2024-04-09/cargo-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:20 [028] 200 https://static.rust-lang.org:443/dist/2024-04-09/cargo-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:20 [030] GET https://static.rust-lang.org:443/dist/2024-04-09/clippy-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:20 [030] 200 https://static.rust-lang.org:443/dist/2024-04-09/clippy-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:21 [032] GET https://static.rust-lang.org:443/dist/2024-04-09/rust-docs-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:21 [032] 200 https://static.rust-lang.org:443/dist/2024-04-09/rust-docs-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:21 [034] GET https://static.rust-lang.org:443/dist/2024-04-09/rust-std-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:21 [034] 200 https://static.rust-lang.org:443/dist/2024-04-09/rust-std-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:22 [036] GET https://static.rust-lang.org:443/dist/2024-04-09/rustc-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:22 [036] 200 https://static.rust-lang.org:443/dist/2024-04-09/rustc-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:24 [038] GET https://static.rust-lang.org:443/dist/2024-04-09/rustfmt-1.77.2-x86_64-unknown-linux-gnu.tar.xz
  proxy | 2024/05/06 16:00:24 [038] 200 https://static.rust-lang.org:443/dist/2024-04-09/rustfmt-1.77.2-x86_64-unknown-linux-gnu.tar.xz

I was very surprised when I saw this, and briefly looked for the Dependabot code that was doing this, before realizing that it's actually Cargo's specified behavior. https://rust-lang.github.io/rustup/overrides.html :

To verify which toolchain is active, you can use rustup show, which will also try to install the corresponding toolchain if the current one has not been installed according to the above rules.

This installation behavior also happens when running ordinary cargo commands. So effectively, for public Rust, this issue is already implemented!