bootstrap/dist.rs hard-coded pathways

[Grommish]

When installing with a --prefix, the bootstrap/dist.rs seems to set hardcoded paths that do not include the set --prefix:

        // Prepare the image directory
        builder.create_dir(&image.join("share/zsh/site-functions"));
        builder.create_dir(&image.join("etc/bash_completion.d"));
        let cargo = builder.ensure(tool::Cargo { compiler, target });
        builder.install(&cargo, &image.join("bin"), 0o755);
        for man in t!(etc.join("man").read_dir()) {
            let man = t!(man);
            builder.install(&man.path(), &image.join("share/man/man1"), 0o644);
        }
        builder.install(&etc.join("_cargo"), &image.join("share/zsh/site-functions"), 0o644);
        builder.copy(&etc.join("cargo.bashcomp.sh"), &image.join("etc/bash_completion.d/cargo"));
        let doc = image.join("share/doc/cargo");

https://github.com/rust-lang/rust/blob/master/src/bootstrap/dist.rs#L1210

This causes errors when /etc, /share, and the like are not available. I am building rustc/cargo from the 1.45.2 source for use with OpenWrt. I couldn't find a missed config.toml entry or configure flag for this.

I am not familar with the rust language, and would not feel comfortable submitting a PR on it. Tail error from the build failure:

Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage2 tool cargo (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.19s
Dist cargo stage2 (x86_64-unknown-linux-gnu)
    finished in 9.372
Install cargo stage2 (Some("x86_64-unknown-linux-gnu"))
install: creating uninstall script at /home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib/uninstall.sh
install: installing component 'cargo'
cp: cannot create regular file '/etc/bash_completion.d/cargo': Permission denied
chmod: cannot access '/etc/bash_completion.d/cargo': No such file or directory
install: error: file creation failed. see logs at '/home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib/install.log'

command did not execute successfully: "sh" "/home/grommish/openwrt/build_dir/hostpkg/rustc-1.45.2/build/tmp/dist/cargo-0.46.1-dev-x86_64-unknown-linux-gnu/install.sh" "--prefix=/home/grommish/openwrt/build_dir/hostpkg/.cargo" "--sysconfdir=/etc" "--datadir=/home/grommish/openwrt/build_dir/hostpkg/.cargo/share" "--docdir=/home/grommish/openwrt/build_dir/hostpkg/.cargo/share/doc/rust" "--bindir=/home/grommish/openwrt/build_dir/hostpkg/.cargo/bin" "--libdir=/home/grommish/openwrt/build_dir/hostpkg/.cargo/lib" "--mandir=/home/grommish/openwrt/build_dir/hostpkg/.cargo/share/man" "--disable-ldconfig"
expected success, got: exit code: 1

failed to run: /home/grommish/openwrt/build_dir/hostpkg/rustc-1.45.2/build/bootstrap/debug/bootstrap install
Build completed unsuccessfully in 0:01:33

install.log:

$ umask 022 && mkdir -p "/home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib"
$ echo "3" > "/home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib/rust-installer-version"
install: creating uninstall script at /home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib/uninstall.sh
$ cp /home/grommish/openwrt/build_dir/hostpkg/rustc-1.45.2/build/tmp/dist/cargo-0.46.1-dev-x86_64-unknown-linux-gnu/install.sh /home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib/uninstall.sh
install: installing component 'cargo'
$ echo "cargo" >> "/home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib/components"
$ umask 022 && mkdir -p "/etc/bash_completion.d"
install: copying file /etc/bash_completion.d/cargo
$ cp /home/grommish/openwrt/build_dir/hostpkg/rustc-1.45.2/build/tmp/dist/cargo-0.46.1-dev-x86_64-unknown-linux-gnu/cargo/etc/bash_completion.d/cargo /etc/bash_completion.d/cargo
$ chmod 644 /etc/bash_completion.d/cargo
install: error: file creation failed. see logs at '/home/grommish/openwrt/build_dir/hostpkg/.cargo/lib/rustlib/install.log'

[ian-h-chamberlain]

I just found this issue, I think there is actually a config.toml entry for this, but for some reason it defaults to absolute path instead of relative, unlike all the other directories. Note sysconfdir in config.toml.example:

# =============================================================================
# General install configuration options
# =============================================================================
[install]

# Instead of installing to /usr/local, install to this path instead.
#prefix = "/usr/local"

# Where to install system configuration files
# If this is a relative path, it will get installed in `prefix` above
#sysconfdir = "/etc"

# Where to install documentation in `prefix` above
#docdir = "share/doc/rust"

# Where to install binaries in `prefix` above
#bindir = "bin"

# Where to install libraries in `prefix` above
#libdir = "lib"

# Where to install man pages in `prefix` above
#mandir = "share/man"

# Where to install data in `prefix` above
#datadir = "share"

Is there a specific reason this is not relative by default?

[Mark-Simulacrum]

At least on my system, the resulting /usr/local/etc seems to be empty, which may suggest that it's an uncommon destination? But I don't think the defaults for these options are necessarily well thought out, so in general a PR that change the whole set to match a tool that's more commonly installed (clang?) would likely be accepted.

[ian-h-chamberlain]

Obviously Linux isn't the only platform, but here is a reference which seems to explain why /usr/local/etc might be desirable: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s09.html

On macOS with brew this seems to be where packages store some configuration, based on this: https://docs.brew.sh/Formula-Cookbook#variables-for-directory-locations

Personally, my expectation would be that if a prefix is set, that the entire installation would be under said prefix. To do otherwise strikes me as a bug, personally.

To your other point - yes, the defaults may not be very well thought out, but at least consistency would be good. So I would argue that perhaps the defaults could be reworked but that making them all relative would be a good first step towards a better default.

[Grommish]

Currently running on 1.57.0..

The issue stems from the fact I set --prefix on ./configure that aren't honored down the line.

MUSL_CONFIGURE_ARGS = \
  --set=target.$(RUSTC_TARGET_ARCH).ar=$(TARGET_AR) \
  --set=target.$(RUSTC_TARGET_ARCH).cc=$(TARGET_CC_NOCACHE) \
  --set=target.$(RUSTC_TARGET_ARCH).cxx=$(TARGET_CXX_NOCACHE) \
  --set=target.$(RUSTC_TARGET_ARCH).linker=$(TARGET_CC_NOCACHE) \
  --set=target.$(RUSTC_TARGET_ARCH).musl-root=$(TOOLCHAIN_DIR) \
  --set=target.$(RUSTC_TARGET_ARCH).ranlib=$(TARGET_RANLIB)

HOST_CONFIGURE_OPTS += CARGO_HOME=$(CARGO_HOME)
RUST_COMMOM_ARGS := \
  --build=$(RUSTC_HOST_ARCH) \
  --dist-compression-formats=xz \
  --enable-full-tools \
  --enable-llvm-link-shared \
  --enable-llvm-plugins \
  --enable-missing-tools \
  --enable-ninja \
  --enable-optimize \
  --enable-optimize-llvm \
  --enable-parallel-compiler \
  --enable-sanitizers \
  --release-channel=nightly \
  ${MUSL_CONFIGURE_ARGS}

HOST_CONFIGURE_ARGS = \
  --target=$(RUSTC_TARGET_ARCH) \
  --host=$(RUSTC_HOST_ARCH) \
  --prefix=$(STAGING_DIR_HOST) \
  --bindir=$(STAGING_DIR_HOST)/bin \
  --libdir=$(STAGING_DIR_HOST)/lib \
  --sysconfdir=$(STAGING_DIR_HOST)/etc \
  --datadir=$(STAGING_DIR_HOST)/share \
  --mandir=$(STAGING_DIR_HOST)/man \
  ${RUST_COMMOM_ARGS}

I set those specifically to override any defaults. However, $(PYTHON) x.py --config ./config.toml install seems to ignore it after I call $(PYTHON) x.py --config ./config.toml dist. Which is fine if you are installing it like most people do. I, however, am installing it to a fakeroot build system.

I gave up on this issue and got around it by manually manipulating the dist packages and calling the install.sh on each with a --prefix call on the install script.

        cd $(RUST_TMP_DIR) && \
          find -iname "*.xz" -exec tar -xJf {} ";" && \
          find ./* -type f -name install.sh -execdir sh {} --prefix=$(STAGING_DIR_HOST) --disable-ldconfig \;

[okeuday]

In the 1.73.0 release, a non-root installation done with configuration as:

./configure --prefix=PREFIX --enable-extended

Will show that dist.rs still has a hard-coded path from the cargo installation at: https://github.com/rust-lang/rust/blob/ae466d2d0a82f460afa6cfeb4b424d82b26e1e6d/src/bootstrap/src/core/build_steps/dist.rs#L1087