search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.38k stars 12.72k forks source link

Building rust 1.81.0 does network access in the "build" phase #130708

Closed he32 closed 1 month ago

he32 commented 1 month ago

I tried to package rust 1.81.0 for pkgsrc-wip, and while trying to do a native build on NetBSD/amd64, I get:

STDERR ----
Updating crates.io index
warning: spurious network error (3 tries remaining): [5] Couldn't resolve proxy name (Could not resolve proxy: downloads-forbidden-except-during-fetch)
warning: spurious network error (2 tries remaining): [5] Couldn't resolve proxy name (Could not resolve proxy: downloads-forbidden-except-during-fetch)
warning: spurious network error (1 tries remaining): [5] Couldn't resolve proxy name (Could not resolve proxy: downloads-forbidden-except-during-fetch)
error: failed to sync
Caused by:
  failed to load pkg lockfile

Caused by:
  failed to get `serde` as a dependency of package `cargo-platform v0.1.8 (/usr/pkgsrc/wip/rust181/work/rustc-1.81.0-src/src/tools/cargo/crates/cargo-platform)`

Caused by:
  download of config.json failed

Caused by:
  failed to download from `https://index.crates.io/config.json`

Caused by:
  [5] Couldn't resolve proxy name (Could not resolve proxy: downloads-forbidden-except-during-fetch)

Traceback (most recent call last):
  File "/usr/pkgsrc/wip/rust181/work/rustc-1.81.0-src/./x.py", line 50, in <module>
    bootstrap.main()
  File "/usr/pkgsrc/wip/rust181/work/rustc-1.81.0-src/src/bootstrap/bootstrap.py", line 1203, in main
    bootstrap(args)
  File "/usr/pkgsrc/wip/rust181/work/rustc-1.81.0-src/src/bootstrap/bootstrap.py", line 1179, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/usr/pkgsrc/wip/rust181/work/rustc-1.81.0-src/src/bootstrap/bootstrap.py", line 186, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /usr/pkgsrc/wip/rust181/work/rustc-1.81.0-src/build/bootstrap/debug/bootstrap -v dist -j 32

I expected to see this happen: no download / network access during the build phase, please.

Instead, this happened: I got the above.

The pkgsrc packaging system insists on checksumming all the build components, all to be downloaded in the "fetch" phase and verified against already-recorded checksums, and therefore sets up an un-resolvable proxy to trap exactly this "anti-packaging" behavior. Doing network access of any type during the build phase is at best considered to be "bad form", as the components which go into the build could then vary depending on external unverifiable factors, possibly causing the build to produce inconsistent results between different builds. At worst this could be a security issue.

This means that the 1.81.0 tarball isn't completely "frozen"...

Hints for working around this gratefully accepted.

bjorn3 commented 1 month ago

Duplicate of https://github.com/rust-lang/rust/issues/130099.

tl;dr: While building the new source tarball as part of the build process, we try to vendor all dependencies even if they have already been vendored. Previously the resulting error when offline would be accidentally ignored, but due to a refactoring of the build system it no longer gets ignored, instead aborting the entire build.

As solution try backporting https://github.com/rust-lang/rust/pull/130110 and set dist.vendor = false in config.toml. (I think --set dist.vendor=false would work if you use ./configure instead of manually creating config.toml.)

workingjubilee commented 1 month ago

duplicate thus

he32 commented 1 month ago

Duplicate of https://github.com/rust-lang/rust/issues/130099.

tl;dr: While building the new source tarball as part of the build process, we try to vendor all dependencies even if they have already been vendored. Previously the resulting error when offline would be accidentally ignored, but due to a refactoring of the build system it no longer gets ignored, instead aborting the entire build.

I find that a little strange. I would instead have expected that the tarball was entirely self-contained, and required no further verification or updating, especially not during the build phase.

As solution try backporting https://github.com/rust-lang/rust/pull/130110 and set dist.vendor = false in config.toml. (I think --set dist.vendor=false would work if you use ./configure instead of manually creating config.toml.)

Thanks, did that, and it works, and I can disable the workaround in the package again.