edgelesssys / contrast

Deploy and manage confidential containers on Kubernetes
https://docs.edgeless.systems/contrast
GNU Affero General Public License v3.0
183 stars 7 forks source link

generated policy mismatch for layers containing non-ASCII symbolic links #624

Closed burgerdev closed 3 months ago

burgerdev commented 3 months ago

Contrast release v0.7.0 contains a bug that prevents starting containers that contain non-ASCII symbolic link names.

Symptom

The symptom for this is that pods are crashlooping with the following message (indicating a failed policy check):

Error: failed to create containerd task: failed to create shim task: "CreateContainerRequest is blocked by policy [...]

Root Cause

The container image layer tarballs are directly mounted as block devices to the guest VM with the tardev-snapshotter. These devices are verified with a dm-verity checksum that's precalculated by contrast generate (specifically, by the embedded genpolicy tool). The precalculation also involves creating an index over the files in the tarball, which mirrors the index created by the snapshotter.

Microsoft's genpolicy tool contains a fix for long link names, which also happens to be triggered by UTF-8 symlink names due to some canonicalization step. The resulting index is different from what the tardev-snapshotter produces, and thus the dm-verity hashes do not match.

Example Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bad-layer
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: bad-layer
  template:
    metadata:
      labels:
        app.kubernetes.io/name: bad-layer
    spec:
      containers:
      - image: docker.io/library/httpd:2.4.59-bookworm@sha256:10182d88d7fbc5161ae0f6f758cba7adc56d4aae2dc950e51d72c0cf68967cea
        name: bad-layer
        resources:
          limits:
            memory: 50Mi
          requests:
            memory: 50Mi
      runtimeClassName: contrast-cc

Problematic symlink:

$ docker run -it --rm docker.io/library/httpd:2.4.59-bookworm ls -l /etc/ssl/certs
[...]
lrwxrwxrwx 1 root root     48 Jun 13 18:28  988a38cb.0 -> 'NetLock_Arany_=Class_Gold=_F'$'\305\221''tan'$'\303\272''s'$'\303\255''tv'$'\303\241''ny.pem'
[...]
lrwxrwxrwx 1 root root     83 Jun 13 18:28 
'NetLock_Arany_=Class_Gold=_F'$'\305\221''tan'$'\303\272''s'$'\303\255''tv'$'\303\241''ny.pem' -> '/usr/share/ca-certificates/mozilla/NetLock_Arany_=Class_Gold=_F'$'\305\221''tan'$'\303\272''s'$'\303\255''tv'$'\303\241''ny.crt'
[...]
katexochen commented 3 months ago

Fix was released with https://github.com/edgelesssys/contrast/releases/tag/v0.7.1