Closed cjdcordeiro closed 11 months ago
This conversation came up today. We should address it. You'll see that I added commentary about ICU in my announcement.
This example Dockerfile demonstrates how one would include ICU to a downstream image:
FROM golang:1.18 as chisel
RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel
FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build
COPY --from=chisel /opt/chisel/chisel /usr/bin/
RUN mkdir /rootfs \
&& chisel cut --release "ubuntu-22.04" --root /rootfs \
libicu70_libs
WORKDIR /source
# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore
# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore
# final stage/image
FROM mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
COPY --from=build /rootfs /
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "dotnetapp.dll"]
I've tested it out with our sample app in dotnet-docker and it works as expected.
Note that due to https://github.com/canonical/chisel/issues/10, it will have result in inefficiencies due to the dependencies that libicu70 has. Essentially, the resulting image ends up with two copies of libc6, libgcc-s1, and libstdc++6.
Therefore, this scenario stresses the importance of having a fix for https://github.com/canonical/chisel/issues/10.
UPDATE: See my next post which describes a workaround for https://github.com/canonical/chisel/issues/10 that allows you to achieve a clean image without duplicate data.
Note that due to https://github.com/canonical/chisel/issues/10, it will have result in inefficiencies due to the dependencies that libicu70 has. Essentially, the resulting image ends up with two copies of libc6, libgcc-s1, and libstdc++6.
I have a workaround for this issue. It's a bit verbose in the Dockerfile, but gives you exactly what you need with 0 bytes of duplicate data in the image.
FROM golang:1.18 as chisel
RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel
FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build
RUN apt-get update \
&& apt-get install -y fdupes \
&& rm -rf /var/lib/apt/lists/*
COPY --from=chisel /opt/chisel/chisel /usr/bin/
COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref
RUN mkdir /rootfs \
&& chisel cut --release "ubuntu-22.04" --root /rootfs \
libicu70_libs \
\
# Remove duplicates from rootfs that exist in runtime-ref
&& fdupes /runtime-ref /rootfs -rdpN \
\
# Delete duplicate symlinks
# Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
&& getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
# Combine set of symlinks between rootfs and runtime-ref
&& (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
# Sort them
| sort \
# Find the duplicates
| uniq -d \
# Extract just the path to the symlink
| cut -d' ' -f1 \
# Prepend the rootfs directory to the paths
| sed -e 's/^/\/rootfs/' \
# Delete the files
| xargs rm \
\
# Delete empty directories
&& find /rootfs -type d -empty -delete
WORKDIR /source
# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore
# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore
# final stage/image
FROM mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
COPY --from=build /rootfs /
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "dotnetapp.dll"]
The key differences from my previous post are in the build stage with these sets of lines:
RUN apt-get update \
&& apt-get install -y fdupes \
&& rm -rf /var/lib/apt/lists/*
COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref
# Remove duplicates from rootfs that exist in runtime-ref
&& fdupes /runtime-ref /rootfs -rdpN \
\
# Delete duplicate symlinks
# Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
&& getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
# Combine set of symlinks between rootfs and runtime-ref
&& (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
# Sort them
| sort \
# Find the duplicates
| uniq -d \
# Extract just the path to the symlink
| cut -d' ' -f1 \
# Prepend the rootfs directory to the paths
| sed -e 's/^/\/rootfs/' \
# Delete the duplicate files
| xargs rm \
\
# Delete empty directories
&& find /rootfs -type d -empty -delete
This cleans up the rootfs in preparation to have it copied into the final stage. It removes all the duplicate files that already exist in the target image.
This leaves you with a very clean final image with no wasted space as can be seen from this output from the dive tool:
┃ ● Layers ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Cmp Size Command
13 MB FROM 5b6f455ba3ead89
73 MB COPY /usr/share/dotnet /usr/share/dotnet # buildkit
0 B COPY /dotnet-symlink /usr/bin # buildkit
35 MB COPY /rootfs / # buildkit
0 B WORKDIR /app
202 kB COPY /app . # buildkit
│ Layer Details ├──────────────────────────────────────────────────────────────
Tags: (unavailable)
Id: 5b6f455ba3ead894517699e52137b1ad3775b1b6d8ef9b85d32b3540ddd2d5e7
Digest: sha256:2d2b3e55f5510eb00f1b3c6769f8e6411a2b7d5238b2880504bd01f01ecd5833
Command:
COPY /rootfs / # buildkit
│ Image Details ├──────────────────────────────────────────────────────────────
Image name: test
Total Image size: 121 MB
Potential wasted space: 0 B
Image efficiency score: 100 %
Count Total Space Path
Compare this to the output of dive if you don't clean up the rootfs directory:
┃ ● Layers ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Cmp Size Command
13 MB FROM 5b6f455ba3ead89
73 MB COPY /usr/share/dotnet /usr/share/dotnet # buildkit
0 B COPY /dotnet-symlink /usr/bin # buildkit
42 MB COPY /rootfs / # buildkit
0 B WORKDIR /app
202 kB COPY /app . # buildkit
│ Layer Details ├──────────────────────────────────────────────────────────────
Tags: (unavailable)
Id: 5b6f455ba3ead894517699e52137b1ad3775b1b6d8ef9b85d32b3540ddd2d5e7
Digest: sha256:2d2b3e55f5510eb00f1b3c6769f8e6411a2b7d5238b2880504bd01f01ecd5833
Command:
COPY /rootfs / # buildkit
│ Image Details ├──────────────────────────────────────────────────────────────
Image name: test
Total Image size: 129 MB
Potential wasted space: 15 MB
Image efficiency score: 94 %
Count Total Space Path
2 4.5 MB /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30
2 4.4 MB /lib/x86_64-linux-gnu/libc.so.6
2 2.1 MB /lib/x86_64-linux-gnu/libmvec.so.1
2 1.9 MB /lib/x86_64-linux-gnu/libm.so.6
2 482 kB /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2 251 kB /lib/x86_64-linux-gnu/libgcc_s.so.1
2 203 kB /lib/x86_64-linux-gnu/libnsl.so.1
2 137 kB /lib/x86_64-linux-gnu/libresolv.so.2
2 113 kB /lib/x86_64-linux-gnu/libc_malloc_debug.so.0
2 88 kB /lib/x86_64-linux-gnu/libnss_compat.so.2
2 80 kB /lib/x86_64-linux-gnu/libthread_db.so.1
2 54 kB /lib/x86_64-linux-gnu/libnss_hesiod.so.2
2 43 kB /lib/x86_64-linux-gnu/libpthread.so.0
2 38 kB /lib/x86_64-linux-gnu/libmemusage.so
2 29 kB /lib/x86_64-linux-gnu/librt.so.1
2 29 kB /lib/x86_64-linux-gnu/libBrokenLocale.so.1
2 29 kB /lib/x86_64-linux-gnu/libpcprofile.so
2 29 kB /lib/x86_64-linux-gnu/libanl.so.1
2 29 kB /lib/x86_64-linux-gnu/libutil.so.1
2 29 kB /lib/x86_64-linux-gnu/libdl.so.2
2 29 kB /lib/x86_64-linux-gnu/libnss_dns.so.2
2 29 kB /lib/x86_64-linux-gnu/libnss_files.so.2
2 0 B /usr/lib/x86_64-linux-gnu/libstdc++.so.6
2 0 B /lib64/ld-linux-x86-64.so.2
But all of this is just a workaround for the chisel tool not being aware of the dependencies that already exist in the target image (https://github.com/canonical/chisel/issues/10). If it was aware of that, the output of chisel would produce a /rootfs directory that is equivalent to what is achieved by this workaround.
cool workaround @mthalman . thanks.
Indeed, https://github.com/canonical/chisel/issues/10 is asking for a broader solution. We already have someone (@rebornplusplus) who's going to start working on it soon.
Is there any progress on official guidance how to include ICU package in Chiseled images?
@kamilzzz the proposed solution from https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1191703196 is the latest guide on this. As you can see from https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1225912006, this is not the ideal solution though, and the latter is a bit too verbose to be included as the official way to achieve this.
We're expecting a new feature in Chisel, in the near future, which migth make this process much cleaner. But until then, https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1225912006 is the way to go.
Today I tried to just rerun the same build pipeline I had for my application with this hacky hack script
So mine Dockerfile was like that
# See details: https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1225912006
FROM golang:1.18 as chisel
RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel
FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build
RUN apt-get update \
&& apt-get install -y fdupes \
&& rm -rf /var/lib/apt/lists/*
COPY --from=chisel /opt/chisel/chisel /usr/bin/
COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref
RUN mkdir /rootfs \
&& chisel cut --release "ubuntu-22.04" --root /rootfs \
libicu70_libs \
\
# Remove duplicates from rootfs that exist in runtime-ref
&& fdupes /runtime-ref /rootfs -rdpN \
\
# Delete duplicate symlinks
# Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
&& getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
# Combine set of symlinks between rootfs and runtime-ref
&& (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
# Sort them
| sort \
# Find the duplicates
| uniq -d \
# Extract just the path to the symlink
| cut -d' ' -f1 \
# Prepend the rootfs directory to the paths
| sed -e 's/^/\/rootfs/' \
# Delete the files
| xargs rm \
\
# Delete empty directories
&& find /rootfs -type d -empty -delete
FROM mcr.microsoft.com/dotnet/nightly/aspnet:6.0.20-jammy-chiseled AS publish
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
WORKDIR /app
COPY --from=build /rootfs /
COPY publish/app .
#New default port for non-root images
EXPOSE 8080
ENTRYPOINT ["dotnet", "app.dll"]
I received such results
(Reading database ... 7953 files and directories currently installed.)
##[error]#15 8.882 Preparing to unpack .../libpcre2-32-0_10.39-3ubuntu0.1_amd64.deb ...
##[error]#15 8.928 Unpacking libpcre2-32-0:amd64 (10.39-3ubuntu0.1) ...
##[error]#15 9.000 Selecting previously unselected package fdupes.
##[error]#15 9.002 Preparing to unpack .../fdupes_1%3a2.1.2-1build1_amd64.deb ...
##[error]#15 9.014 Unpacking fdupes (1:2.1.2-1build1) ...
##[error]#15 9.101 Setting up libpcre2-32-0:amd64 (10.39-3ubuntu0.1) ...
##[error]#15 9.123 Setting up fdupes (1:2.1.2-1build1) ...
##[error]#15 9.184 Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
##[error]#15 DONE 9.5s
##[error]#14 [chisel 4/4] RUN go build ./cmd/chisel
##[error]#14 6.689 # github.com/canonical/chisel/internal/archive
##[error]#14 6.689 internal/archive/credentials.go:219:16: undefined: errors.Join
##[error]#14 6.689 note: module requires Go 1.20
##[error]#14 ERROR: process "/bin/sh -c go build ./cmd/chisel" did not complete successfully: exit code: 2
##[error]------
##[error] > [chisel 4/4] RUN go build ./cmd/chisel:
##[error]0.378 go: downloading gopkg.in/yaml.v3 v3.0.0-20220512140231-539c8e751b99
##[error]0.383 go: downloading go.starlark.net v0.0.0-20220328144851-d1966c6b9fcd
##[error]0.398 go: downloading github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb
##[error]0.398 go: downloading github.com/klauspost/compress v1.15.4
##[error]0.463 go: downloading github.com/ulikunitz/xz v0.5.10
##[error]0.508 go: downloading golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1
##[error]1.216 go: downloading golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
##[error]6.689 # github.com/canonical/chisel/internal/archive
##[error]6.689 internal/archive/credentials.go:219:16: undefined: errors.Join
##[error]6.689 note: module requires Go 1.20
##[error]------
##[error]Dockerfile:6
##[error]--------------------
##[error] 4 | RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
##[error] 5 | WORKDIR /opt/chisel
##[error] 6 | >>> RUN go build ./cmd/chisel
##[error] 7 |
##[error] 8 |
##[error]--------------------
##[error]ERROR: failed to solve: process "/bin/sh -c go build ./cmd/chisel" did not complete successfully: exit code: 2
##[error]The process '/bin/docker' failed with exit code 1
When I changed Dockerfile to FROM golang:1.20 as chisel
all worked as expected. Is that OK that build without any changes from my side in the code of my applications and even with pinned images to exact versions of images (I only not used sha pinning but seems that that would not help here too :) that would build binaries for me would not work just after 2 weeks I implemented that?
@cjdcordeiro @mthalman
hi @Bafff
this can indeed happen since this recipe is building the Chisel binary from its git HEAD. We had to recently change our Dockerfiles in a similar manner -> https://github.com/ubuntu-rocks/dotnet/commit/0957b3905b2d56db34613b2e56a837086d20a719
To avoid this, you can:
checkout
a specific Chisel commit before building the binaryNote: We're planning on producing Microsoft images with icu built-in shortly. That said, making chisel easier to use to add ICU or other libraries remains important.
@cjdcordeiro, this issue has been open for quite some time and there's an increasing interest in Chiseled images now. While we have Chiseled images with ICU built-in, we're still interested in guidance for how to install additional packages on top of an existing chiseled Ubuntu installation. Are there any plans for that?
hi @lbussell .
This is a generic chiselling topic, and we'll be working on creating such documentation over the next few months. For now, I have added some .net-specific guides on how to do that, in https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image.
I hope that helps ;)
I have added some .net-specific guides on how to do that, in https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image.
Thanks! Will this work with chisel-wrapper in order to generate a merged dpkg status file? We'd like to retain the ability to accurately scan images with tools like Trivy, etc.
I'm afraid not. The wrapper generates a dpkg/status file only for the given slices in the corresponding command. Until the Chisel DB is merged, I'm afraid you'd have to merge the contents. Something like (using the example from https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image):
...
COPY --from=mcr.microsoft.com/dotnet/runtime-deps:8.0.0-jammy-chiseled-amd64 /var/lib/dpkg/status /original-dpkg-status
RUN chisel-wrapper --generate-dpkg-status /additional-dpkg-status ... && \
cat /additional-dpkg-status /original-dpkg-status > /merged-dpkg-status
...
# FINAL IMAGE
...
COPY --from=base /merged-dpkg-status /var/lib/dpkg/status
It isn't super clean, but our efforts are currently going to the Chisel DB as we believe that's the right path.
Our containers depend on the geoipupdate utility to keep the MaxMind database fresh on the container. How can we install the deb package in our image that is based on chiseled image? When we have the RUN dpkg -i /geoipupdate.deb
in the docker file, we get a failed to solve: process "/bin/sh -c dpkg -i /geoipupdate.deb" did not complete successfully
error during build.
Hi @johnwc .
Mixing debs and slices on the same image can be tricky because the installation of debs relies on package managers and other system dependencies, which is one of the things chisel removes.
My suggestion is: propose the slice definitions for that package in https://github.com/canonical/chisel-releases/, and then install those pkg slices alongside the other ones.
If you really need the deb, then you may try to run dpkg -x
on a different build stage, and copy those contents onto the final stage, alongside the chiselled contents. You'll not have that pkg listed in dpkg/status
though, unless you amend it by hand.
I created a request for a guide in the chisel repo. https://github.com/canonical/chisel/issues/118
libicu might be needed for some use cases, but we're not including it in these chiseled images.
we should then document the process (with an example) of adding libicu to the existing runtime* ROCKs