canonical / chisel

GNU Affero General Public License v3.0
270 stars 42 forks source link

Guide on adding third-party packages to chiseled base image. #118

Open johnwc opened 8 months ago

johnwc commented 8 months ago

Our containers depend on the geoipupdate utility to keep the MaxMind database fresh on the container. How can we install the deb package in our image that is based on chiseled image? When we have the RUN dpkg -i /geoipupdate.deb in the docker file, we get a failed to solve: process "/bin/sh -c dpkg -i /geoipupdate.deb" did not complete successfully error during build. This also relies on cron to run the utility on a schedule.

rebornplusplus commented 8 months ago

Hi John. The geoipupdate package seems to be in the Ubuntu archives as well: listed here. If you are planning to install from the Ubuntu archives, it can be chiselled. Looking at the contents, it shouldn't probably be tough either to do so. You can then install the slices directly using chisel.

However, if you are planning on using the deb downloaded from the Github Release, I am afraid there is no mechanism to chisel that deb. If you want to install the deb on top of your chiselled image, you might need dpkg in there. So you will probably need to chisel dpkg first. And then you might be able to install the deb on top of your file system.

And for cron, you can chisel the package and install it in your chiselled file system.

Please let me know what you plan to do and if you have more questions. Cheers.

johnwc commented 8 months ago

Where are the steps to know how to chisel a package from the ubuntu archive? How do you find out what slices are in a package?

The main problem with using dpkg using a docker RUN command, is that it attempts to use a shell to execute the command.

lbussell commented 8 months ago

Where are the steps to know how to chisel a package from the ubuntu archive? How do you find out what slices are in a package?

The issue you linked to contains some guidance in this comment: https://github.com/ubuntu-rocks/dotnet/issues/21#issuecomment-1815937307

The main problem with using dpkg using a docker RUN command, is that it attempts to use a shell to execute the command.

You should make sure you are using the "exec" form for RUN instructions and not the "shell" form when using distroless/Chiseled images.

johnwc commented 8 months ago

@lbussell it does not answer my question, on how you find out the names to use for slices in the chisel command.

rebornplusplus commented 8 months ago

There are no predefined rules to be honest. Typically, we list the libraries (examples: /usr/lib/../*.so, /lib/../*.so) under the libs slice, binaries in bins and config files (DEBIAN/conffiles inside the deb lists those) in config. In most cases, you might find yourself doing the same.

You might also need to define some slices of your own if such cases arise. For example, if you are building a very minimal slice which contains the very minimal files yet works, you might put those inside a core slice as done in python3.11. As long as you are slicing your packages logically, it should be fine. One extreme example of logically grouping the files inside a package would be the libpython3.11-standard slices.

It would be best if you skim over some existing slice definition files available in the chisel-releases repo and take inspirations from there.

johnwc commented 8 months ago

Ok, so slice yaml files are manually created by hand, they are not generated with a utility? If we create custom slices, how do we get chisel to see them and use them? Is there a specific location it looks for all slices?

rebornplusplus commented 8 months ago

Ok, so slice yaml files are manually created by hand, they are not generated with a utility?

Yup, they are manually created by hand, as of now. We did think about an utility which suggests slices, but it's not in the plans yet.

If we create custom slices, how do we get chisel to see them and use them? Is there a specific location it looks for all slices?

Ideally, you would want your slices to end up in the chisel-releases repo. Then you can use the --release option to specify which ubuntu version (branch names in that repo) you want to use for packages. If, however, they are not merged in the repo, you can point the --release option to a local directory which contains the slices in slices/ and a chisel.yaml file.

yevheniilavrenchuk commented 8 months ago

Hi @rebornplusplus, I already asked here, but want to fix it faster, maybe you know the solution.

Is there a way to use slice configs on ubuntu-22.04, which are written for ubuntu-23.04? When I just copied all nodejs dependent config files it didn't work, it is telling that one of the dependencies is missing in archive.

...
16.22 2024/02/02 12:37:14 Fetching ubuntu 22.04 jammy-security suite details...
17.27 2024/02/02 12:37:15 Release date: Fri, 02 Feb 2024 11:25:15 UTC
17.27 2024/02/02 12:37:15 Fetching index for ubuntu 22.04 jammy-security main component...
19.46 2024/02/02 12:37:17 Fetching index for ubuntu 22.04 jammy-security universe component...
21.70 2024/02/02 12:37:20 Fetching ubuntu 22.04 jammy-updates suite details...
22.79 2024/02/02 12:37:21 Release date: Fri, 02 Feb 2024 11:26:07 UTC
22.79 2024/02/02 12:37:21 Fetching index for ubuntu 22.04 jammy-updates main component...
25.50 2024/02/02 12:37:24 Fetching index for ubuntu 22.04 jammy-updates universe component...
27.66 error: slice package "node-undici" missing from archive
rebornplusplus commented 8 months ago

Oh hi again, replied there!