paritytech / substrate

Substrate: The platform for blockchain innovators
Apache License 2.0
8.39k stars 2.65k forks source link

docker: Substrate Docker image error building binaries #14804

Open ltfschoen opened 1 year ago

ltfschoen commented 1 year ago

Is there an existing issue?

Experiencing problems? Have you tried our Stack Exchange first?

Description of bug

If I follow the instructions in the Substrate repository README here that link to the Substrate Builder Docker Image instructions here, the instructions say it might only include the following binaries:

Then when I run ./build.sh it outputs the following error:

show error ``` Building parity/substrate:latest docker image, hang on! [+] Building 2602.6s (14/15) docker:desktop-linux => [internal] load build definition from substrate_builder.Dockerfile 0.0s => => transferring dockerfile: 1.88kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 136B 0.0s => [internal] load metadata for docker.io/library/ubuntu:20.04 5.0s => [internal] load metadata for docker.io/paritytech/ci-linux:production 3.1s => [builder 1/4] FROM docker.io/paritytech/ci-linux:production@sha256:dc9abf9e877c5bad94828245406dac8a186530e1ad6a1b5f2072e5e19e1f64b4 0.0s => CACHED [stage-1 1/6] FROM docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba 0.0s => [internal] load build context 0.6s => => transferring context: 46.95MB 0.6s => CACHED [builder 2/4] WORKDIR /substrate 0.0s => CACHED [builder 3/4] COPY . /substrate 0.0s => CACHED [builder 4/4] RUN cargo build --locked --release 0.0s => CACHED [stage-1 2/6] COPY --from=builder /substrate/target/release/substrate /usr/local/bin 0.0s => ERROR [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin 0.0s => ERROR [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin 0.0s => ERROR [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin 0.0s ------ > [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin: ------ ------ > [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin: ------ ------ > [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin: ------ substrate_builder.Dockerfile:21 -------------------- 19 | COPY --from=builder /substrate/target/release/subkey /usr/local/bin 20 | COPY --from=builder /substrate/target/release/node-template /usr/local/bin 21 | >>> COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin 22 | 23 | RUN useradd -m -u 1000 -U -s /bin/sh -d /substrate substrate && \ -------------------- ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 4b02f120-ba24-4d54-b2be-a38dc6dce883::kor4q7o2o99zc2mscfob9bp7v: "/substrate/target/release/chain-spec-builder": not found real 43m22.870s user 0m4.775s sys 0m4.250s ```

Although the error appears obvious now, initially I thought it was because I was using a new Apple M2 chip, but it was actually just because for some reason that Docker approach only generates the substrate binary, but not the subkey, node-template, or chain-spec-builder binaries like it used to as well.

I believe the solution is to either: 1) Change line 6 of substrate_builder.Dockerfile to instead run the following to generate all the binaries in the /substrate/target/release directory, so it doesn't generate an error when they are used later on lines 19-21:

RUN cargo build --locked --release && \
    cargo build -p subkey --release --locked && \
    cargo build -p node-template --release --locked && \
    cargo build -p chain-spec-builder --release --locked

2) Alternatively remove lines 19-21 of substrate_builder.Dockerfile so it only builds and copies across the substrate binary to the Docker container. The benefit of doing this is that it should reduce the overall build time for those getting started with Substrate, as most users may only want to use the substrate binary. We could then update the Substrate Builder Docker Image instructions here to mention that if the user wants to use the subkey, node-template, or chain-spec-builder binaries too then they can simply enter the Substrate Docker container shell with the command docker run -it -d --hostname substrate --name helios parity/substrate && docker exec -it substrate /bin/bash where they could then run the substrate binary with ./target/release/substrate --help and manually generate the other binaries by running the following to build those other binaries:

cargo build -p subkey --release --locked && \
cargo build -p node-template --release --locked && \
cargo build -p chain-spec-builder --release --locked

and then run the following to create symlinks

ln -s /usr/local/bin/subkey /substrate/target/release/subkey
ln -s /usr/local/bin/node-template /substrate/target/release/node-template
ln -s /usr/local/bin/chain-spec-builder /substrate/target/release/chain-spec-builder

so they could then run those other binaries with:

./target/release/subkey --help
./target/release/node-template --help
./target/release/chain-spec-builder --help

or exit the Docker container and run the follow as it already mentions is available in the README

./run.sh subkey --help
./run.sh node-template --version
./run.sh chain-spec-builder --help

Steps to reproduce