Permission errors when running deepfence-cluster-agent containers on OKD (community Openshift)

[rjeczkow]

Describe the bug I'm trying to install deepfence agent from the helm chart (https://artifacthub.io/packages/helm/threatmapper/deepfence-agent/2.3.1) on OKD 4.15 cluster.

While the cluster-agent pods are running fine (their containers are privileged), I'm having troubles with deepfence-cluster-agent pods (containers are not privileged).

This is the log from deepfence-cluster-agent pod:

sudo: effective uid is not 0, is /usr/bin/sudo on a file system with the 'nosuid' option set or an NFS file system without root privileges?
Starting periodic command scheduler: croncron: can't open or create /var/run/crond.pid: Permission denied
failed!
chmod: changing permissions of '/etc/logrotate.d/fenced_logrotate.conf': Operation not permitted
/var/spool/cron/: mkstemp: Permission denied
Traceback (most recent call last):
File "/usr/bin/supervisord", line 33, in <module>
sys.exit(load_entry_point('supervisor==4.2.2', 'console_scripts', 'supervisord')())
File "/usr/lib/python3/dist-packages/supervisor/supervisord.py", line 359, in main
go(options)
File "/usr/lib/python3/dist-packages/supervisor/supervisord.py", line 369, in go
d.main()
File "/usr/lib/python3/dist-packages/supervisor/supervisord.py", line 72, in main
self.options.make_logger()
File "/usr/lib/python3/dist-packages/supervisor/options.py", line 1494, in make_logger
loggers.handle_file(
File "/usr/lib/python3/dist-packages/supervisor/loggers.py", line 419, in handle_file
handler = RotatingFileHandler(filename, 'a', maxbytes, backups)
File "/usr/lib/python3/dist-packages/supervisor/loggers.py", line 213, in __init__
FileHandler.__init__(self, filename, mode)
File "/usr/lib/python3/dist-packages/supervisor/loggers.py", line 160, in __init__
self.stream = open(filename, mode)
PermissionError: [Errno 13] Permission denied: '/var/log/supervisor/supervisord.log'
Starting agent

Helm chart is creating a DaemonSet deepfence-agent and a Deployment deepfence-agent-cluster.

• The DaemonSet pods (deepfence-agent) are running privileged containers hence the pod's scc is privileged

$ oc get pod -l name=deepfence-agent -o yaml | grep scc
  openshift.io/scc: privileged

• The Deployment pods (deepfence-cluster-agent) by default is not running privileged container (the pod's scc is restricted-v2)

$ oc get pod -l name=deepfence-cluster-agent -o yaml | grep scc
      openshift.io/scc: restricted-v2

The cluster is not being registered in the Deepfence management console.

To Reproduce

Install https://artifacthub.io/packages/helm/threatmapper/deepfence-agent/2.3.1#install on OKD cluster (I tried 4.11, 4.14, 4.15)

Expected behavior All pods should run without errors, the cluster should be registered in the Deepfence management console.

Workaround When the deepfence-cluster-agent deployment's containers are privileged everything works as expected.

The deployment can be patched as follows:

oc patch deployment deepfence-cluster-agent -p '{"spec":{"template":{"spec":{"containers":\[{"name":"deepfence-cluster-agent","securityContext":{"privileged":true}}\]}}}}'

Then the cluster is successfully being registered in the Deepfence management console.

[ramanan-ravi]

Hello @rjeczkow @Kajot-dev

New deepfence/deepfence-agent helm chart version: 2.3.2

[rjeczkow]

Thank you!