search

vgwidt / sumi

Multi-user issue tracking and knowledge base app built with Yew & Actix
MIT License
10 stars 1 forks source link

Improve Docker builds #11

Open vgwidt opened 1 year ago

vgwidt commented 1 year ago

Docker build speeds can be sped up significantly.

Separate Dockerfiles are not necessary either, especially once caching is properly utilized.

vgwidt commented 1 year ago

If using pre-compiled binaries don't forget to install wget:

RUN apt-get update && apt-get install -y wget && rm -rf /var/lib/apt/lists/*
vgwidt commented 1 year ago

Threw together a sample that doesn't cache cargo dependencies yet.

FROM rust:latest as base
RUN apt-get update && apt-get install -y wget && rm -rf /var/lib/apt/lists/*
RUN wget -qO- https://github.com/thedodd/trunk/releases/download/v0.16.0/trunk-x86_64-unknown-linux-gnu.tar.gz | tar -xzf- && chmod +x trunk && cp trunk /usr/local/cargo/bin/
RUN rustup target add wasm32-unknown-unknown
RUN wget -qO- https://github.com/rustwasm/wasm-bindgen/releases/download/0.2.84/wasm-bindgen-0.2.84-aarch64-unknown-linux-gnu.tar.gz | tar -xzf- && cp ./wasm-bindgen-0.2.84-aarch64-unknown-linux-gnu/wasm-bindgen /usr/local/cargo/bin/ && chmod +x /usr/local/cargo/bin/wasm-bindgen

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi-backend

I'm not sure it is actually worth avoiding using cargo to install trunk and wasm-bindgen as that bit should be cached.

vgwidt commented 1 year ago

Sample with cargo-chef for caching project (so far only backend) dependencies:

FROM rust:latest as base
RUN cargo install cargo-chef
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as planner
WORKDIR /usr/src/sumi
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM base as cacher
WORKDIR /usr/src/sumi
COPY --from=planner /usr/src/sumi/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
COPY --from=cacher /usr/src/sumi/target target
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY ./backend/diesel.toml ./backend/diesel.toml
COPY ./certificates ./certificates
COPY ./dist ./dist
COPY ./backend/migrations ./backend/migrations
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

This builds trunk and wasm-bindgen as that stage will be cached anyways. Only the backend dependencies are cached; not sure how to handle frontend yet. Using the Rust image and building Diesel results in a 1.5GB image as there are no precompiled binaries readily available (diesel 2379).

vgwidt commented 1 year ago

COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi doesn't appear to be working.

vgwidt commented 1 year ago

COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi doesn't appear to be working.

Dist folder wasn't being copied from the builder, need to change COPY ./dist ./dist to COPY --from=builder /usr/src/sumi/dist ./dist.

Need to fix up .dockerignore as well, and can just copy . . on final stage. It'll copy unused source files but they use minimal space.

vgwidt commented 1 year ago

A related issue is that some .env variables are still required for building the frontend.

vgwidt commented 1 year ago

Merged 7ac1b16ec84829cfbeb219ad4bdbc6c387dd8594 which includes the new Dockerfile. It currently only caches the backend.

Environment variables are now written to the index.html file when the backend is run. I'll need to figure out a solution that allows de-coupling of the API and web server, though that is currently not possible at this point anyways.

vgwidt commented 1 year ago

The issue with caching the frontend has to do with Cargo workspace limitations. We can tell the frontend to target wasm32-unknown-unknown by adding the following in .cargo/config.toml:

[build]
target = "wasm32-unknown-unknown"

However, when building from the root workspace folder, it does not look at this config file for members. Alternatively, Cargo nightly allows us to force the package target in the member Cargo.toml: https://github.com/rust-lang/cargo/issues/9406. I tested it and it works great, but I don't want to depend on nightly.

It appears that the only other option is to move frontend out of the workspace (would need to do an exclude or eliminate workspace altogether).

vgwidt commented 1 year ago

It looks like I might be able to build frontend deps using config.toml while building in the frontend folder. Sample (not working) Dockerfile that might demonstrate how it could be done when building with Docker:

FROM rust:latest as base
RUN cargo install cargo-chef
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./Cargo.lock
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release
WORKDIR /usr/src/sumi
RUN rm -rf ./backend/src && rm -rf ./frontend/src

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
COPY --from=cacher /usr/src/sumi/target target
RUN touch -a -m ./backend/src/main.rs && touch -a -m ./frontend/src/lib.rs
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi
vgwidt commented 1 year ago

Missed root Cargo.file in last comment. The following works and built in less than 2 minutes with everything cached:

FROM rust:latest as base
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./Cargo.lock
COPY ./Cargo.toml ./Cargo.toml
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release
WORKDIR /usr/src/sumi
RUN rm -rf ./backend/src && rm -rf ./frontend/src

FROM base as builder
WORKDIR /usr/src/sumi
COPY --from=cacher /usr/src/sumi/target target
COPY . .
RUN touch -a -m ./backend/src/main.rs && touch -a -m ./frontend/src/lib.rs
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

I attempted to get it working the same way with cargo-chef without success. I may try it again as it is a pretty clean solution.

It can be cleaned up a bit (i.e. remove RUN rm).

vgwidt commented 1 year ago

Slightly cleaned up version without touch, as I don't think we need it:

FROM rust:latest as base
RUN cargo install cargo-chef
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./
COPY ./Cargo.toml ./
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release

FROM base as builder
WORKDIR /usr/src/sumi
COPY --from=cacher /usr/src/sumi/target target
COPY . .
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

It would be nice to not have to rebuild the both if only one was changed. Is this because Docker ignores cache after change was detected at COPY . .?

vgwidt commented 1 year ago

Touch is needed otherwise it may use the builds made in the cacher stage. Removed cargo-chef.

I'm pretty satisfied with the current state, but I still would like to get a smaller final image. That would require building diesel_cli by ourselves.

FROM rust:latest as base
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./
COPY ./Cargo.toml ./
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
COPY --from=cacher /usr/src/sumi/target target
RUN touch -a -m ./backend/src/main.rs && touch -a -m ./frontend/src/lib.rs
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi
vgwidt commented 1 year ago

Addition of shared broke the Dockerbuild, fixed in 5d7778c

vgwidt commented 1 year ago

https://docs.rs/diesel_migrations/latest/diesel_migrations/macro.embed_migrations.html can be used in lieu of diesel_cli. It reads the migrations at compile time.

vgwidt commented 1 year ago

Dependency caching is not working very well. I find myself rebuilding them most of the time without any changes to Cargo.toml or Cargo.lock.

vgwidt commented 1 year ago

In case I want to continue to use diesel_cli but build in builder stage:

In builder:

RUN cargo install diesel_cli --no-default-features --features postgres

In final image:

RUN apt-get -y update
RUN apt-get install -y libpq-dev
COPY --from=builder /usr/local/cargo/bin/diesel /usr/bin/diesel

libssl.so.1.1 doesn't exist in the bookworm image, but it does in the Rust, so we need to either move it or build it. It might look like this:

RUN apt-get install -y wget
RUN wget https://www.openssl.org/source/openssl-1.1.1o.tar.gz && tar -zxvf openssl-1.1.1o.tar.gz
WORKDIR ./openssl-1.1.1o
RUN ./config && make && make test
RUN mv libcrypto.so.1.1 /usr/bin/ && mv libssl.so.1.1 /usr/bin/