siderolabs / talos

Talos Linux is a modern Linux distribution built for Kubernetes.
https://www.talos.dev
Mozilla Public License 2.0
5.76k stars 466 forks source link

support AppArmor profiles for containers #8715

Open smira opened 2 months ago

smira commented 2 months ago

See conversation here for details: https://github.com/siderolabs/talos/pull/8641#issuecomment-2074597065

konrader commented 1 month ago

I started looking at how I could create a Talos system extension that contributes /usr/sbin/apparmor_parser to rootfs (where containerd looks it). I managed to create a basic Docker build file that builds apparmor_parser from source:

FROM ubuntu:24.04

RUN apt update && apt install -y git make autoconf-archive pkg-config libtool g++ gettext bison flex

RUN git clone -b apparmor-3.1 https://gitlab.com/apparmor/apparmor.git

WORKDIR /apparmor/libraries/libapparmor
RUN ./autogen.sh
RUN ./configure
RUN make

WORKDIR /apparmor/parser
RUN make

FROM ubuntu:24.04

COPY --from=0 /apparmor/parser/apparmor_parser /usr/sbin/apparmor_parser

Now I'm a bit lost around how I get this into a Talos extension that I can try out on a test Talos cluster.

smira commented 1 month ago

I don't think we allow extensions to modify /usr/sbin, only /usr/local/... atm.

We are considering this feature still, but so far working on other things, so no active work happening here.

frezbo commented 1 month ago

Also since talos is based on musl, you'd have to build using an example in the extensions repo, probably look at nut-client

smira commented 1 month ago

But if kubelet/CRI hardcodes the apparmor_parser location, we'd have to make it part of standard Talos install, and then the question is how big it is (including whatever dependencies it has)

konrader commented 1 month ago

Unfortunately containerd has the path to apparmor_parser hardcoded: https://github.com/containerd/containerd/blob/release/1.7/pkg/apparmor/apparmor_linux.go#L38

The tricky part I see in containerd is that it automatically applies a default AppArmor profile to all started containers (if they don't have any security spec specified) if it has detected that host supports AppArmor (that checks both kernel support and that userland tool is available). The sorry part is that it does not allow starting a container with AppArmor security spec referencing already loaded profile if the host AppArmor support check returned false, even if in this case the apparmor_parser userland tool is not needed (profile reference is passed down to runc). I almost feel like this should be an issue raised with containerd project.

konrader commented 1 month ago

Also since talos is based on musl, you'd have to build using an example in the extensions repo, probably look at nut-client

Here is a Dockerfile for compiling apparmor_parser against musl.

FROM alpine:3.19.1

RUN apk add -U --no-cache git g++ automake autoconf-archive pkgconfig make libtool gettext bison flex python3 vim sed musl-libintl linux-headers bash

RUN git clone -b apparmor-3.1 https://gitlab.com/apparmor/apparmor.git

WORKDIR /apparmor/libraries/libapparmor
RUN ./autogen.sh
RUN ./configure
RUN make

WORKDIR /apparmor/parser
RUN ../common/list_af_names.sh > base_af_names.h
RUN make

FROM alpine:3.19.1

COPY --from=0 /apparmor/parser/apparmor_parser /sbin/apparmor_parser

But I'm not sure if I should convert this into Talos extension package since it does not allow copying things to /sbin.

konrader commented 1 month ago

The actual path for where containerd looks for apparmor_parser is /sbin/apparmor_parser (https://github.com/containerd/containerd/blob/release/1.7/pkg/apparmor/apparmor_linux.go#L38). Not sure this impacts anything, but can a Talos extension contribute files to /sbin/?

Size of the musl compiled /sbin/apparmor_parser is 8.2 MB.

smira commented 1 month ago

This can't be done via a system extension so far, the plan is to put that to core Talos probably (not sure if it needs more stuff like libraries).

If you could try to submit a build to pkgs, that would be a great first step. Even if it doesn't quite work, we could fix it up.

konrader commented 2 weeks ago

Since https://github.com/siderolabs/pkgs/pull/979 is merged, I added the required lines in Makefile and Dockerfile in this repo to include apparmor package when building Talos. It seems to work fine. You want me to do a PR for it or what is your process of including a new package in the Talos build?

frezbo commented 2 weeks ago

Since siderolabs/pkgs#979 is merged, I added the required lines in Makefile and Dockerfile in this repo to include apparmor package when building Talos. It seems to work fine. You want me to do a PR for it or what is your process of including a new package in the Talos build?

yes please