Closed GitarPlayer closed 9 months ago
This makes sense. I will try to look into this. Perhaps it's just correcting the RBAC yamls that get created by the Helm chart.
I would love to help, I tried installing the nifi operator on Red Hat OpenShift Local (https://developers.redhat.com/products/openshift-local/overview) with no success. When I ran make run it would return some error that the tls certificates were not found. But it was the first time for me to build a go operator with make so maybe I am also just a noop ;-). I followed this guide: https://konpyutaika.github.io/nifikop/docs/6_contributing/1_developer_guide I managed to install the operator on a test OpenShift cluster (I did have to edit the runAsUser and fsGroup of the nifikop pod so it would be in the namespace range by hand (see point 2).
Thanks for that run-down! Were you able to actually get the operator installed successfully after these steps? Or are you still stuck?
Adding the /finalizers
RBAC permission on all of the objects is a fairly easy thing for me to add, but i'm curious if you were able to resolve any of these issues.
I did manage to install the operator but I did not manage to install a nifi cluster yet through the operator. I will try again sometime this or next week, then I can maybe do a PR with the exact kubebuilder annotations for RBAC and maybe some OpenShift related DOC.
That'd be great! I'm also happy to help whip up these changes (kube builder annotations + helm chart Role
mods). If you aren't able to get to it, sharing a working Role
including all of the appropriate permissions needed ought to be good enough for me to go in and diff it all.
I'm admittedly very green when it comes to OpenShift (solely a Rancher-k8s user), but i should be able to help troubleshoot.
So I did some research and edited the RBAC kubebuilder annotations in my fork: https://github.com/konpyutaika/nifikop/compare/master...GitarPlayer:nifikop:master According to here: https://developers.redhat.com/blog/2020/09/11/5-tips-for-developing-kubernetes-operators-with-the-new-operator-sdk#tip_3__discovering_the_cluster_you_re_running_on
I set the runAsUser and fsGroup parameters in the CRD of config/samples/simplenificluster.yaml so it would run inside my nifi namespace. But it could not create the cluster because of filesystem permission issues:
oc logs -f $(oc get po -l app=nifi -l nodeId=1 -n nifi --no-headers -o custom-columns=":metadata.name") -n nifi nifi
cp: cannot create regular file '/opt/nifi/nifi-current/conf/bootstrap-notification-services.xml': Permission denied
cp: cannot create regular file '/opt/nifi/nifi-current/conf/bootstrap.conf': Permission denied
cp: cannot create regular file '/opt/nifi/nifi-current/conf/logback.xml': Permission denied
cp: cannot create regular file '/opt/nifi/nifi-current/conf/login-identity-providers.xml': Permission denied
cp: cannot create regular file '/opt/nifi/nifi-current/conf/nifi.properties': Permission denied
cp: cannot create regular file '/opt/nifi/nifi-current/conf/state-management.xml': Permission denied
cp: cannot create regular file '/opt/nifi/nifi-current/conf/zookeeper.properties': Permission denied
This is from the nifi container and not from the zookeper container logs. This is not a nifikop operator problem because the image that is being used by the nifi container apache/nifi (https://github.com/apache/nifi/blob/main/nifi-docker/dockerhub/Dockerfile) has filepermission set that it only works for the user 1000. Sorry for that terrible formatting.
[ezra@U337659:/tmp/nifikop]$ podman run -d apache/nifi a1ae52de18dc191166ecf8595652c5a782d6295a7b360903bee95c11f645167e
[ezra@U337659:/tmp/nifikop]$ podman exec -it -l
nifi@a1ae52de18dc:/opt/nifi/nifi-current$ ll total 360 drwxrwxr-x 1
nifi nifi 4096 Apr 15 09:23 ./ drwxr-xr-x 1 nifi nifi 4096 Apr 10 20:05 ../ drwxrwxr-x 1 nifi nifi 4096 Apr 3 21:28 bin/ drwxrwxr-x 2 nifi nifi 4096 Apr 15 09:23 conf/ drwxr-xr-x 2 nifi nifi 4096 Apr 10 20:05 content_repository/ drwxr-xr-x 2 nifi nifi 4096 Apr 10 20:05 database_repository/ drwxrwxr-x 3 nifi nifi 4096 Apr 3 21:28 docs/ drwxrwxr-x 2 nifi nifi 4096 Apr 3 21:28 extensions/ drwxr-xr-x 2 nifi nifi 4096 Apr 10 20:05 flowfile_repository/ drwxrwx--- 6 nifi nifi 12288 Apr 3 21:28 lib/ -rw-r--r-- 1 nifi nifi 175405 Apr 3 21:28 LICENSE drwxr-xr-x 2 nifi nifi 4096 Apr 15 09:23 logs/ -rw-r--r-- 1 nifi nifi 110857 Apr 3 21:28 NOTICE drwxr-xr-x 2 nifi nifi 4096 Apr 10 20:05 provenance_repository/ -rw-r--r-- 1 nifi nifi 4935 Apr 3 21:28 README drwxr-xr-x 2 nifi nifi 4096 Apr 15 09:23 run/ drwxr-xr-x 2 nifi nifi 4096 Apr 10 20:05 state/ drwxr-xr-x 4 nifi nifi 4096 Apr 15 09:23 work/
nifi@a1ae52de18dc:/opt/nifi/nifi-current$ id
uid=1000(nifi) gid=1000(nifi) groups=1000(nifi)
So there are two options:
1. reference a custom built container image
2. ask the apache/nifi team to adjust filesystem permissions
I already mailed them regarding this matter but I am not sure they will do it since it is quite particular to OpenShift to have guid 0 rwX permissions on the relevant files. They would profit greatly if they would use an image like ubi9/openjdk-11-runtime as a base since this is regularly patched and scanned for vulnerabilities and minimal in size.
I am not 100% certain that these RBAC fixes make it work entirely, I would first have to build a OpenShift compatible nifi image and verify if the rest works smoothly as well. I will report this as soon as I know more.
I built all nifikop compatible images here: https://quay.io/repository/rhnclimocha/nifi?tab=tags. They are based on ubi9/openjdk-11-runtime and are built on a CRON schedule to get security patches for the OS. As soon I have time I will try if nifikop works with the fixed RBAC and the custom docker image.
I managed to get the simplenifi cluster running on OpenShift by using my own images quay.io/rhnclimocha/nifi:1.7.0 and setting the fsGroup and runAsUser settings particular to my nifi namespace. I will do a PR request that outlines all the steps for it to work on OpenShift. Could you merge the updated RBAC kubebuilder annotations?
Apologies! I was out last week at KubeCon. I'll jump back into this :)
Thanks for your patience.
Is your feature request related to a problem?
Currenlty nifikop does work with OpenShift due to missing RBAC permissions on finalizers: See here #https://github.com/konpyutaika/nifikop/issues/69
There are also some additional changes needed like for example the CRD NodeConfigGroups needs to be extended and more stuff. My question is if I implement these changes will they be merged or not?
Describe the solution you'd like to see
Nifikop runs on OpenShift and the getting started page explains how to get up and running with the specific OpenShift gotchas. Maybe nifikop even becomes a OpenShift certified operator but let's start small :-).
Describe alternatives you've considered
The Nifikop team has no interest in that (fair enough) and I consider a different nifi k8s project to make OpenShift friendly.
Additional context
I tried installing it on the latest OpenShift and it fails. I got 70% of the fixes to make it work on OpenShift and I wonder where to go with this.