1st draft keylime conf, payloads and service files

[mpeters]

• Rearranged keylime confs to be in the deployment/ folder under subdirs for target nodes
• Auto run action on openshift-worker nodes just installs /tmp/keylime-approved
• Node failure over revocation action issues oc commands (that will fail for now until I figure out how to get keylime to know the k8s node names)
• Add service files for okd nodes for
  • tpm_server
  • tpm2-abrmd
  • keylime_ima_emulator
  • keylime_agent
• Add service files for keylime-controller for
  • tpm_server
  • tpm2-abrmd
  • keylime_ima_emulator
  • keylime_agent
  • keylime_registrar
  • keylime_verifier

[mpeters]

Did we ever figure out the container filesystem problem? For your videos you were using /etc and /boot which containers running on the open shift nodes probably won't trigger in IMA. /usr/bin and /usr/lib64 will definitely overlap with the containers' filesystems and trigger ima checks.

[lukehinds]

Did we ever figure out the container filesystem problem? For your videos you were using /etc and /boot which containers running on the open shift nodes probably won't trigger in IMA. /usr/bin and /usr/lib64 will definitely overlap with the containers' filesystems and trigger ima checks.

We should be able to increase (by reducing the entries in excludes.txt). I think we should be good as long as we ignore stuff in var/* which is where a lot of cgroups and volumes will be mounted by the container on the host. I was chatting to another RH associate about this and he thinks it unlikely we would be able to change the hosts files in /usr/sbin etc as that would be a security hole. I can work on this with you if you like, we can do a hangouts session.

[mpeters]

Did we ever figure out the container filesystem problem? For your videos you were using /etc and /boot which containers running on the open shift nodes probably won't trigger in IMA. /usr/bin and /usr/lib64 will definitely overlap with the containers' filesystems and trigger ima checks.

We should be able to increase (by reducing the entries in excludes.txt). I think we should be good as long as we ignore stuff in var/* which is where a lot of cgroups and volumes will be mounted by the container on the host. I was chatting to another RH associate about this and he thinks it unlikely we would be able to change the hosts files in /usr/sbin etc as that would be a security hole. I can work on this with you if you like, we can do a hangouts session.

@lukehinds I still think there's some miscommunication about what's happening and I've probably explained it poorly. It's not that the container is overwriting files on the system, and it's not the container volumes and cgroups in /var that are causing problems. It's that when containers access a file that is scoped to their view/namespace IMA gets confused about what the actual path of the file is and thus the whitelist will never work. Maybe a concrete example will explain it better:

On the worker-0 node (worker-mrbw7) there exists a file with path /usr/lib64/libexpat.so.1 (it's actually a symlink, but that doesn't matter for this) with a SHA256 hash of 4a1fb990caa2a03186568589e62dafc601845e922ea67287cc0e51f2dbf123ee. If our whitelist contains this file (or it's not excluded) everything is good.

Next, OC runs a lot of containers behind the scenes, one example is prometheus-k8s-1. This container has it's own namespaced filesystem and also contains a file with the path /usr/lib64/libexpat.so.1 . But this container is running completely different libraries so it's SHA256 sum is not the same, but rather ea08eb0db35f42f5dd62f113c65f9622359a45cf356f1fa730b23b804f9968bd. When the container runs somehow it tells IMA that it's running a file with path /usr/lib64/libexpat.so.1 with one hash, but the whitelist expects another because the path /usr/lib64/libexpat.so.1 on the host has a different one. This triggers an integrity measurement error and thus fails the revocation.

Nothing on the host has been changed, meaning no security hole. But IMA is confused about the path of the file because IMA doesn't appear to be container/namespace aware. This same pattern repeats for hundreds of file paths on the system that might overlap with something in container. I don't know if this something a change in the ima_policy could fix or a real bug in IMA.

Is that explanation clearer?