LukeMathWalker / cargo-chef

A cargo-subcommand to speed up Rust Docker builds using Docker layer caching.
Apache License 2.0
1.72k stars 113 forks source link

Strange binary executing behavior #192

Closed bangbaew closed 1 year ago

bangbaew commented 1 year ago

This is my Dockerfile:

FROM lukemathwalker/cargo-chef:latest-rust-latest AS chef
WORKDIR /app
ARG MOLD_VERSION=1.10.1

# download_url=$(curl -s https://api.github.com/repos/rui314/mold/releases/latest | grep "mold-.*x86_64-linux.tar.gz" | cut -d '"' -f 4) wget -q $download_url -O mold.tar.gz
# Download the mold linker archive
RUN wget -q https://github.com/rui314/mold/releases/download/v${MOLD_VERSION}/mold-${MOLD_VERSION}-x86_64-linux.tar.gz -O mold.tar.gz

# Extract the mold executable
RUN mkdir -p mold
RUN tar -zxf mold.tar.gz --strip-component=2 --wildcards mold-*-x86_64-linux/bin/mold
RUN chmod +x mold
RUN mv mold /usr/bin/ld

# Build only dependencies to speed up subsequent builds
ADD Cargo.toml Cargo.lock ./
RUN mkdir -p src
RUN echo "fn main() {}" > src/main.rs
RUN cargo fetch

FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS cooker
COPY --from=planner /app/recipe.json recipe.json
# Build dependencies - this is the caching Docker layer!
RUN cargo chef cook --release --recipe-path recipe.json

FROM cooker AS generator
COPY .cargo .cargo
COPY prisma prisma
COPY src/bin src/bin <--- Problem part
RUN cargo gen-release <--- alias of: cargo run --release --bin prisma generate

FROM cooker AS builder
# Build application
COPY src src <--- Problem part
COPY --from=generator /app/src/app/db/prisma.rs /app/src/app/db/prisma.rs
RUN cargo build --release --bin main
RUN strip target/release/main
RUN readelf -p .comment target/release/main

# We do not need the Rust toolchain to run the binary!
FROM frolvlad/alpine-glibc AS runner
RUN apk update
RUN apk add libssl1.1
COPY --from=builder /app/target/release/main /
ENTRYPOINT ["/main"]
  1. When I started building an image, if I use COPY . ., everything works fine, but if my source code changes a little bit, even the change is not relevant (eg. docker-compose, Dockerfile, or github pipeline), it has to recompile everything over again and the wait time is around 20 minutes minimum, so I'll copy only the necessary files for each process.
  2. Running Prisma binary in release mode takes 5 minutes to finish the process, but running in debug takes only few seconds, I don't know why this is happening (but it has to compile the debug dependencies too, I wanna use the cooked release dependencies), So I use COPY prisma prisma which contains needed Prisma schema for generating Prisma client instead of COPY . . so I can save 5 minutes from caching if my schema doesn't change.
  3. After COPY src/bin src/bin or COPY src/bin/prisma.rs src/bin/prisma.rs, I tried executing cargo run --release --bin prisma generate, the terminal showed this, and nothing happened. image
  4. So I tried copying the same file (prisma.rs) from the host into the container manually. image
  5. Replacing prisma.rs inside the container with one from host. image
  6. Tried running cargo run --release --bin prisma generate and it worked fine, the Prisma cli started as expected. (Opening the file and save it again also works) image

The current workaround is to replace COPY src/bin/prisma.rs src/bin/prisma.rs with RUN echo "fn main() {prisma_client_rust_cli::run();}" > src/bin/prisma.rs or simply add RUN cat src/bin/prisma.rs > temp && mv temp src/bin/prisma.rs after the COPY command and it works great, but same thing happens in the builder layer, where I use COPY src src, it would never compile, I had to add RUN cat src/main.rs > temp && mv temp src/main.rs, but I think there should be a better workaround where I don't have to replace a file with itself.

If I don't use cargo-chef, everything works normally without having to replace those files manually, so I guess this issue is related to cargo-chef, please someone fix it.

LukeMathWalker commented 1 year ago

To use cargo-chef as intended, the Dockerfile needs to look like this:

FROM lukemathwalker/cargo-chef:latest-rust-latest AS chef
WORKDIR /app
ARG MOLD_VERSION=1.10.1

# download_url=$(curl -s https://api.github.com/repos/rui314/mold/releases/latest | grep "mold-.*x86_64-linux.tar.gz" | cut -d '"' -f 4) wget -q $download_url -O mold.tar.gz
# Download the mold linker archive
RUN wget -q https://github.com/rui314/mold/releases/download/v${MOLD_VERSION}/mold-${MOLD_VERSION}-x86_64-linux.tar.gz -O mold.tar.gz

# Extract the mold executable
RUN mkdir -p mold
RUN tar -zxf mold.tar.gz --strip-component=2 --wildcards mold-*-x86_64-linux/bin/mold
RUN chmod +x mold
RUN mv mold /usr/bin/ld

FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS cooker
COPY --from=planner /app/recipe.json recipe.json
COPY . .
# This builds the dependencies, you shouldn't be doing that manually
RUN cargo chef cook --release --recipe-path recipe.json

FROM cooker AS generator
RUN cargo gen-release <--- alias of: cargo run --release --bin prisma generate

FROM cooker AS builder
RUN cargo build --release --bin main
RUN strip target/release/main
RUN readelf -p .comment target/release/main

# We do not need the Rust toolchain to run the binary!
FROM frolvlad/alpine-glibc AS runner
RUN apk update
RUN apk add libssl1.1
COPY --from=builder /app/target/release/main /
ENTRYPOINT ["/main"]