search

ubuntu-rocks / dotnet

Ubuntu ROCKs for the .NET runtime and family
Apache License 2.0
50 stars 4 forks source link

add use case-specific doc for libicu #21

Closed cjdcordeiro closed 10 months ago

cjdcordeiro commented 2 years ago

libicu might be needed for some use cases, but we're not including it in these chiseled images.

we should then document the process (with an example) of adding libicu to the existing runtime* ROCKs

richlander commented 2 years ago

This conversation came up today. We should address it. You'll see that I added commentary about ICU in my announcement.

mthalman commented 2 years ago

This example Dockerfile demonstrates how one would include ICU to a downstream image:

FROM golang:1.18 as chisel

RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel

FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build

COPY --from=chisel /opt/chisel/chisel /usr/bin/
RUN mkdir /rootfs \
    && chisel cut --release "ubuntu-22.04" --root /rootfs \
        libicu70_libs

WORKDIR /source

# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore

# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled

ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

COPY --from=build /rootfs /
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "dotnetapp.dll"]

I've tested it out with our sample app in dotnet-docker and it works as expected.

Note that due to https://github.com/canonical/chisel/issues/10, it will have result in inefficiencies due to the dependencies that libicu70 has. Essentially, the resulting image ends up with two copies of libc6, libgcc-s1, and libstdc++6.

Therefore, this scenario stresses the importance of having a fix for https://github.com/canonical/chisel/issues/10.

UPDATE: See my next post which describes a workaround for https://github.com/canonical/chisel/issues/10 that allows you to achieve a clean image without duplicate data.

mthalman commented 2 years ago

Note that due to https://github.com/canonical/chisel/issues/10, it will have result in inefficiencies due to the dependencies that libicu70 has. Essentially, the resulting image ends up with two copies of libc6, libgcc-s1, and libstdc++6.

I have a workaround for this issue. It's a bit verbose in the Dockerfile, but gives you exactly what you need with 0 bytes of duplicate data in the image.

FROM golang:1.18 as chisel

RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel

FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build

RUN apt-get update \
    && apt-get install -y fdupes \
    && rm -rf /var/lib/apt/lists/*

COPY --from=chisel /opt/chisel/chisel /usr/bin/
COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref

RUN mkdir /rootfs \
    && chisel cut --release "ubuntu-22.04" --root /rootfs \
        libicu70_libs \
    \
    # Remove duplicates from rootfs that exist in runtime-ref
    && fdupes /runtime-ref /rootfs -rdpN \
    \
    # Delete duplicate symlinks
    # Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
    && getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
    # Combine set of symlinks between rootfs and runtime-ref
    && (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
        # Sort them
        | sort \
        # Find the duplicates
        | uniq -d \
        # Extract just the path to the symlink
        | cut -d' ' -f1 \
        # Prepend the rootfs directory to the paths
        | sed -e 's/^/\/rootfs/' \
        # Delete the files
        | xargs rm \
    \
    # Delete empty directories
    && find /rootfs -type d -empty -delete

WORKDIR /source

# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore

# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled

ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

COPY --from=build /rootfs /
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "dotnetapp.dll"]

The key differences from my previous post are in the build stage with these sets of lines:

RUN apt-get update \
    && apt-get install -y fdupes \
    && rm -rf /var/lib/apt/lists/*
COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref
    # Remove duplicates from rootfs that exist in runtime-ref
    && fdupes /runtime-ref /rootfs -rdpN \
    \
    # Delete duplicate symlinks
    # Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
    && getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
    # Combine set of symlinks between rootfs and runtime-ref
    && (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
        # Sort them
        | sort \
        # Find the duplicates
        | uniq -d \
        # Extract just the path to the symlink
        | cut -d' ' -f1 \
        # Prepend the rootfs directory to the paths
        | sed -e 's/^/\/rootfs/' \
        # Delete the duplicate files
        | xargs rm \
    \
    # Delete empty directories
    && find /rootfs -type d -empty -delete

This cleans up the rootfs in preparation to have it copied into the final stage. It removes all the duplicate files that already exist in the target image.

This leaves you with a very clean final image with no wasted space as can be seen from this output from the dive tool:

┃ ● Layers ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Cmp   Size  Command
     13 MB  FROM 5b6f455ba3ead89
     73 MB  COPY /usr/share/dotnet /usr/share/dotnet # buildkit
       0 B  COPY /dotnet-symlink /usr/bin # buildkit
     35 MB  COPY /rootfs / # buildkit
       0 B  WORKDIR /app
    202 kB  COPY /app . # buildkit

│ Layer Details ├──────────────────────────────────────────────────────────────

Tags:   (unavailable)
Id:     5b6f455ba3ead894517699e52137b1ad3775b1b6d8ef9b85d32b3540ddd2d5e7
Digest: sha256:2d2b3e55f5510eb00f1b3c6769f8e6411a2b7d5238b2880504bd01f01ecd5833
Command:
COPY /rootfs / # buildkit

│ Image Details ├──────────────────────────────────────────────────────────────

Image name: test
Total Image size: 121 MB
Potential wasted space: 0 B
Image efficiency score: 100 %

Count   Total Space  Path

Compare this to the output of dive if you don't clean up the rootfs directory:

┃ ● Layers ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Cmp   Size  Command
     13 MB  FROM 5b6f455ba3ead89
     73 MB  COPY /usr/share/dotnet /usr/share/dotnet # buildkit
       0 B  COPY /dotnet-symlink /usr/bin # buildkit
     42 MB  COPY /rootfs / # buildkit
       0 B  WORKDIR /app
    202 kB  COPY /app . # buildkit

│ Layer Details ├──────────────────────────────────────────────────────────────

Tags:   (unavailable)
Id:     5b6f455ba3ead894517699e52137b1ad3775b1b6d8ef9b85d32b3540ddd2d5e7
Digest: sha256:2d2b3e55f5510eb00f1b3c6769f8e6411a2b7d5238b2880504bd01f01ecd5833
Command:
COPY /rootfs / # buildkit

│ Image Details ├──────────────────────────────────────────────────────────────

Image name: test
Total Image size: 129 MB
Potential wasted space: 15 MB
Image efficiency score: 94 %

Count   Total Space  Path
    2        4.5 MB  /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30
    2        4.4 MB  /lib/x86_64-linux-gnu/libc.so.6
    2        2.1 MB  /lib/x86_64-linux-gnu/libmvec.so.1
    2        1.9 MB  /lib/x86_64-linux-gnu/libm.so.6
    2        482 kB  /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
    2        251 kB  /lib/x86_64-linux-gnu/libgcc_s.so.1
    2        203 kB  /lib/x86_64-linux-gnu/libnsl.so.1
    2        137 kB  /lib/x86_64-linux-gnu/libresolv.so.2
    2        113 kB  /lib/x86_64-linux-gnu/libc_malloc_debug.so.0
    2         88 kB  /lib/x86_64-linux-gnu/libnss_compat.so.2
    2         80 kB  /lib/x86_64-linux-gnu/libthread_db.so.1
    2         54 kB  /lib/x86_64-linux-gnu/libnss_hesiod.so.2
    2         43 kB  /lib/x86_64-linux-gnu/libpthread.so.0
    2         38 kB  /lib/x86_64-linux-gnu/libmemusage.so
    2         29 kB  /lib/x86_64-linux-gnu/librt.so.1
    2         29 kB  /lib/x86_64-linux-gnu/libBrokenLocale.so.1
    2         29 kB  /lib/x86_64-linux-gnu/libpcprofile.so
    2         29 kB  /lib/x86_64-linux-gnu/libanl.so.1
    2         29 kB  /lib/x86_64-linux-gnu/libutil.so.1
    2         29 kB  /lib/x86_64-linux-gnu/libdl.so.2
    2         29 kB  /lib/x86_64-linux-gnu/libnss_dns.so.2
    2         29 kB  /lib/x86_64-linux-gnu/libnss_files.so.2
    2           0 B  /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    2           0 B  /lib64/ld-linux-x86-64.so.2

But all of this is just a workaround for the chisel tool not being aware of the dependencies that already exist in the target image (https://github.com/canonical/chisel/issues/10). If it was aware of that, the output of chisel would produce a /rootfs directory that is equivalent to what is achieved by this workaround.

cjdcordeiro commented 2 years ago

cool workaround @mthalman . thanks.

Indeed, https://github.com/canonical/chisel/issues/10 is asking for a broader solution. We already have someone (@rebornplusplus) who's going to start working on it soon.

kamilzzz commented 1 year ago

Is there any progress on official guidance how to include ICU package in Chiseled images?

cjdcordeiro commented 1 year ago

@kamilzzz the proposed solution from https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1191703196 is the latest guide on this. As you can see from https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1225912006, this is not the ideal solution though, and the latter is a bit too verbose to be included as the official way to achieve this.

We're expecting a new feature in Chisel, in the near future, which migth make this process much cleaner. But until then, https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1225912006 is the way to go.

Bafff commented 1 year ago

Today I tried to just rerun the same build pipeline I had for my application with this hacky hack script

So mine Dockerfile was like that

# See details: https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1225912006
FROM golang:1.18 as chisel

RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel

FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build

RUN apt-get update \
    && apt-get install -y fdupes \
    && rm -rf /var/lib/apt/lists/*

COPY --from=chisel /opt/chisel/chisel /usr/bin/
COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref

RUN mkdir /rootfs \
    && chisel cut --release "ubuntu-22.04" --root /rootfs \
        libicu70_libs \
    \
    # Remove duplicates from rootfs that exist in runtime-ref
    && fdupes /runtime-ref /rootfs -rdpN \
    \
    # Delete duplicate symlinks
    # Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
    && getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
    # Combine set of symlinks between rootfs and runtime-ref
    && (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
        # Sort them
        | sort \
        # Find the duplicates
        | uniq -d \
        # Extract just the path to the symlink
        | cut -d' ' -f1 \
        # Prepend the rootfs directory to the paths
        | sed -e 's/^/\/rootfs/' \
        # Delete the files
        | xargs rm \
    \
    # Delete empty directories
    && find /rootfs -type d -empty -delete

FROM mcr.microsoft.com/dotnet/nightly/aspnet:6.0.20-jammy-chiseled AS publish

ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

WORKDIR /app

COPY --from=build /rootfs /
COPY publish/app .

#New default port for non-root images
EXPOSE 8080

ENTRYPOINT ["dotnet", "app.dll"]

I received such results

(Reading database ... 7953 files and directories currently installed.)
##[error]#15 8.882 Preparing to unpack .../libpcre2-32-0_10.39-3ubuntu0.1_amd64.deb ...
##[error]#15 8.928 Unpacking libpcre2-32-0:amd64 (10.39-3ubuntu0.1) ...
##[error]#15 9.000 Selecting previously unselected package fdupes.
##[error]#15 9.002 Preparing to unpack .../fdupes_1%3a2.1.2-1build1_amd64.deb ...
##[error]#15 9.014 Unpacking fdupes (1:2.1.2-1build1) ...
##[error]#15 9.101 Setting up libpcre2-32-0:amd64 (10.39-3ubuntu0.1) ...
##[error]#15 9.123 Setting up fdupes (1:2.1.2-1build1) ...
##[error]#15 9.184 Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
##[error]#15 DONE 9.5s
##[error]#14 [chisel 4/4] RUN go build ./cmd/chisel
##[error]#14 6.689 # github.com/canonical/chisel/internal/archive
##[error]#14 6.689 internal/archive/credentials.go:219:16: undefined: errors.Join
##[error]#14 6.689 note: module requires Go 1.20
##[error]#14 ERROR: process "/bin/sh -c go build ./cmd/chisel" did not complete successfully: exit code: 2
##[error]------
##[error] > [chisel 4/4] RUN go build ./cmd/chisel:
##[error]0.378 go: downloading gopkg.in/yaml.v3 v3.0.0-20220512140231-539c8e751b99
##[error]0.383 go: downloading go.starlark.net v0.0.0-20220328144851-d1966c6b9fcd
##[error]0.398 go: downloading github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb
##[error]0.398 go: downloading github.com/klauspost/compress v1.15.4
##[error]0.463 go: downloading github.com/ulikunitz/xz v0.5.10
##[error]0.508 go: downloading golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1
##[error]1.216 go: downloading golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
##[error]6.689 # github.com/canonical/chisel/internal/archive
##[error]6.689 internal/archive/credentials.go:219:16: undefined: errors.Join
##[error]6.689 note: module requires Go 1.20
##[error]------
##[error]Dockerfile:6
##[error]--------------------
##[error]   4 |     RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
##[error]   5 |     WORKDIR /opt/chisel
##[error]   6 | >>> RUN go build ./cmd/chisel
##[error]   7 |     
##[error]   8 |     
##[error]--------------------
##[error]ERROR: failed to solve: process "/bin/sh -c go build ./cmd/chisel" did not complete successfully: exit code: 2
##[error]The process '/bin/docker' failed with exit code 1

When I changed Dockerfile to FROM golang:1.20 as chisel all worked as expected. Is that OK that build without any changes from my side in the code of my applications and even with pinned images to exact versions of images (I only not used sha pinning but seems that that would not help here too :) that would build binaries for me would not work just after 2 weeks I implemented that?

@cjdcordeiro @mthalman

cjdcordeiro commented 1 year ago

hi @Bafff

this can indeed happen since this recipe is building the Chisel binary from its git HEAD. We had to recently change our Dockerfiles in a similar manner -> https://github.com/ubuntu-rocks/dotnet/commit/0957b3905b2d56db34613b2e56a837086d20a719

To avoid this, you can:

richlander commented 1 year ago

Note: We're planning on producing Microsoft images with icu built-in shortly. That said, making chisel easier to use to add ICU or other libraries remains important.

https://github.com/dotnet/dotnet-docker/discussions/4821

lbussell commented 10 months ago

@cjdcordeiro, this issue has been open for quite some time and there's an increasing interest in Chiseled images now. While we have Chiseled images with ICU built-in, we're still interested in guidance for how to install additional packages on top of an existing chiseled Ubuntu installation. Are there any plans for that?

cjdcordeiro commented 10 months ago

hi @lbussell .

This is a generic chiselling topic, and we'll be working on creating such documentation over the next few months. For now, I have added some .net-specific guides on how to do that, in https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image.

I hope that helps ;)

lbussell commented 10 months ago

I have added some .net-specific guides on how to do that, in https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image.

Thanks! Will this work with chisel-wrapper in order to generate a merged dpkg status file? We'd like to retain the ability to accurately scan images with tools like Trivy, etc.

cjdcordeiro commented 10 months ago

I'm afraid not. The wrapper generates a dpkg/status file only for the given slices in the corresponding command. Until the Chisel DB is merged, I'm afraid you'd have to merge the contents. Something like (using the example from https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image):

...
COPY --from=mcr.microsoft.com/dotnet/runtime-deps:8.0.0-jammy-chiseled-amd64 /var/lib/dpkg/status /original-dpkg-status
RUN chisel-wrapper --generate-dpkg-status /additional-dpkg-status ... && \
        cat /additional-dpkg-status /original-dpkg-status > /merged-dpkg-status
...
# FINAL IMAGE
...
COPY --from=base /merged-dpkg-status /var/lib/dpkg/status

It isn't super clean, but our efforts are currently going to the Chisel DB as we believe that's the right path.

johnwc commented 7 months ago

Our containers depend on the geoipupdate utility to keep the MaxMind database fresh on the container. How can we install the deb package in our image that is based on chiseled image? When we have the RUN dpkg -i /geoipupdate.deb in the docker file, we get a failed to solve: process "/bin/sh -c dpkg -i /geoipupdate.deb" did not complete successfully error during build.

cjdcordeiro commented 7 months ago

Hi @johnwc .

Mixing debs and slices on the same image can be tricky because the installation of debs relies on package managers and other system dependencies, which is one of the things chisel removes.

My suggestion is: propose the slice definitions for that package in https://github.com/canonical/chisel-releases/, and then install those pkg slices alongside the other ones.

If you really need the deb, then you may try to run dpkg -x on a different build stage, and copy those contents onto the final stage, alongside the chiselled contents. You'll not have that pkg listed in dpkg/status though, unless you amend it by hand.

johnwc commented 7 months ago

I created a request for a guide in the chisel repo. https://github.com/canonical/chisel/issues/118