jlesage
commented
1 year ago Do you get the same failure every time you (re)start the container ? Is v23.03.1 the last version that work fine ?
Open louhisuo opened 1 year ago
jlesage
commented
1 year ago Do you get the same failure every time you (re)start the container ? Is v23.03.1 the last version that work fine ?
louhisuo
commented
1 year ago Yes, I get same failure always when trying run it with setting 'privileged: false'. I had been using v23.03.1 until this point and that was working fine and did not need 'Privileged' mode.
jlesage
commented
1 year ago Can you try other versions (https://github.com/jlesage/docker-firefox/releases) to see the latest one that works? I don't see any change that would explain the sudden change of behaviour.
louhisuo
commented
1 year ago Sure, off course. Will probably do it tomorrow as it is pretty late for my timezone. What comes my mind is that perhaps nothing has really changed from docker-firefox side but perhaps they have tighten up security in TrueNAS / Truecharts side. So what I am wondering if docker-firefox container is e.g. getting sufficient Linux capabilities from Kubernetes.
This is what is currently given for containers
` securityContext: allowPrivilegeEscalation: true capabilities: add:
louhisuo
commented
1 year ago Made some testing and docker-firefox also failed with older releases, even with v23.03.1 which worked for me previously. However I made finding that if I add all 'Capabilities' listed for Baseline profile then docker-firefox container starts and run successfully. So with more relax capabilities also v23.05.2 starts and run.
These profiles are described in Kubernetes documentations under Pod Security Standards
My guess is that TruenNAS/Truechart folks may have tighten up security (which is good thing so kudos for them) when introducing their new common helm chart + custom-app chart which are used to automatically generate deployment manifest for docker-firefox and this caused issue.
louhisuo
commented
1 year ago Some further testing with Linux capabilities to better understand what exact capability set docker-firefox really needs to launch and run.
Truecharts are assigning following capabilities for every pod and deny rest (I did not try to remove any of capabilities from this list, even that is very easy thing to do).
However this is not sufficient for docker-firefox which requires two additional Linux capabilities to launch and run
So it looks like there is no issue in docker-firefox itself which works like charm on Kubernetes when it gets sufficient Linux capabilities. However I would be grateful if you could consider to document which Linux capabilities docker-firefox really need to launch and run?
jlesage
commented
1 year ago However I would be grateful if you could consider to document which Linux capabilities docker-firefox really need to launch and run?
Documenting the minimal set of capabilities might be difficult (I guess we need to try to see what is needed/missing), especially considering that they might change after updates. In general, the container is tested to work properly with capabilities provided by Docker.
louhisuo
commented
1 year ago Fully understood that this is probably difficult to document and may change in the future. I however wanted to make you aware that above seven Linux capabilities were needed to able to launch and run docker-firefox (v23.05.2) successfully on Kubernetes (others were dropped).
Kubernetes lists three different security profiles, including Linux capabilities in their documentation under Pod Security Standards (wondering if Docker has similar documentation?) From there I took 'Baseline' profile as input for my test.
When considering documentation, it is perhaps sufficient just to refer 'Pod Security Standards' Baseline profile as it provides superset of capabilities which docker-firefox needs. This is because Kubernetes documentation considers capabilities listed under Baseline profile "safe" i.e. preventing known privilege escalations. What do you think?
I will remove 'Bug' label from this issue now as this is definitely not bug in docker-firefox but merely Kubernetes configuration issue. But I will be more than happy to test things (as I have Kubernetes setup with docker-firefox) and also exchange ideas how to document this if you think it is worth to do :)
louhisuo
commented
1 year ago Oh, seems that I am not able to remove 'Bug' tag from this issue, my apologies. Please feel free to remove 'Bug' tag.
Current Behavior
Running jlesage/docker-firefox image in TrueNAS Scale kubernetes cluster. My previous docker-firefox version (v23.03.1) deployed in that environment was running nicely without 'Privileged' mode permission but now we I tried to deploy the latest docker-firefox version (v23.05.2) it requires 'Privileged' mode to be set to be started and running without restarting continuously. I have seccomp profile defined for the pod in both releases so I suspect issue being elsewhere. I have attached 'kubectl logs' which prints container logs for both cases. Main difference what I can see that xvnc will not run in failed case.
Expected Behavior
docker-firefox pod starts and runs without enabling 'Privileged' mode.
Steps To Reproduce
Deploy docker-firefox container (v23.05.2) in TrueNAS Scale using its custom-app helm chart which automatically generates deployment manifest for the container and also injects configurations like seccomp profile to pod template. Try to run container without 'Privileged' mode, it starts but enters cycling pod restart.
Environment
Container creation
See Container inspect which shows Kubernetes deployment manifest generated when creating pod thru GUI.
Container log
Container inspect
Anything else?
No response