search

jenkinsci / kubernetes-operator

Kubernetes native Jenkins Operator
https://jenkinsci.github.io/kubernetes-operator
Other
591 stars 231 forks source link

OpenShift Compatibility for Jenkins Operator #826

Open syanpriyajot opened 1 year ago

syanpriyajot commented 1 year ago

Describe the solution you'd like Since v0.6.0, it was mentioned that Jenkins Operator is not compatible with OpenShift. https://github.com/jenkinsci/kubernetes-operator/releases/tag/v0.6.0

Describe alternatives you've considered Is there working documentation for OpenShift using which I can test the compatibility of latest release on OpenShift? Or should exactly the same documentation as for Kubernetes work on OpenShift as well?

@brokenpip3

brokenpip3 commented 1 year ago

Generally speaking I don't think we have something specific that will let the operator to not run correctly in openshift. I believe that in the past the team supported the jenkins version shipped by redhat which is different from the mainline jenkins lts that we use in our operator, so probably the removal was related to that.

Coming back to ocp: I belive that we may have some issue if we do not run the jenkins pod with an scc that will grants the anyuid, or the specific jenkins image id to be set as running user of the container. I'm not sure, we need to test it to understand if my guess will be true. But apart of this I do not expect any other critical issue in running vs ocp.

Now the real problem is: testing. Minishift is stuck at ocp 3.11, we need to find a way to test it via a github actions in ocp 4.x. Will try to build something after a small spring break

kumadee commented 1 year ago

Generally speaking I don't think we have something specific that will let the operator to not run correctly in openshift. I believe that in the past the team supported the jenkins version shipped by redhat which is different from the mainline jenkins lts that we use in our operator, so probably the removal was related to that.

Coming back to ocp: I belive that we may have some issue if we do not run the jenkins pod with an scc that will grants the anyuid, or the specific jenkins image id to be set as running user of the container. I'm not sure, we need to test it to understand if my guess will be true. But apart of this I do not expect any other critical issue in running vs ocp.

Now the real problem is: testing. Minishift is stuck at ocp 3.11, we need to find a way to test it via a github actions in ocp 4.x. Will try to build something after a small spring break

@brokenpip3 I don't think RedHat will support minishift for ocp 4.x any longer. They have moved on to https://github.com/crc-org/crc for ocp 4.x, due to lot of changes in installation.

brokenpip3 commented 1 year ago

yes I saw that but the crc will require 16gb of memory to run and the github runner cannot handle more than 7gb (and we do no have any free credits here to use the enterprise runners). So we need to find a way to test the installation in openshift, if someone has other ideas I can take a look on ocp 4.x and fix all the issues but I cannot run it freely anywhere :(

kumadee commented 1 year ago

I followed the issue https://github.com/crc-org/crc/issues/557 and it led to this issue https://github.com/crc-org/crc/issues/2507, which aims to reduce the resource requirements of OCP 4.8. I think we are blocked until this issue has some meaningful progress.

There were some experimental changes done by crc team https://github.com/crc-org/snc/pull/241 to do this but doesn't look ready for our use here.

brokenpip3 commented 1 year ago

If someone can try to deploy it in openshift and send here the logs, decribe, evetns will be great, I can take a look and try a fix if needed (90% will be a scc issue)

akilada commented 8 months ago

@brokenpip3 I have tried to implement jenkins operator v0.60 on openshift 4.11. However, the jenkins instance pod keeps getting terminated.

The following are my deployment steps:

However, the jenkins instance pod keeps getting terminated soon after initiation. The following are the logs from jenkins operator pod.

jenkins-operator-v0.60.log events.log jenkins-master-describe.log jenkins-operator-describe.log

brokenpip3 commented 8 months ago

@akilada could you please try by using the latest version (0.8.0)? the 0.6.0 it's a very old version and contains issues that make impossible to run it everywhere

BeastHook commented 2 months ago

@brokenpip3 I have tried it with the latest operator version on a 4.13 Openshift cluster. The master pod keeps terminating.

I used the example CR in your doc.

Here is the log of the terminated pod: jenkins-example-jenkins-jenkins-master.log

The rest looks similar to @akilada logs

jonesbusy commented 2 months ago

This is similar to deployment of the helm chart on Openshift/OKD

You just need to adapt some environment vars for the plugin manager tool to download to a writtable location

Like https://github.com/jenkinsci/helm-charts/issues/506#issuecomment-1050898826

brokenpip3 commented 2 months ago

in our case it's slightly better you can choose to:

In any cases we should change our helmchart, or even better the code to always use /tmp or a specific emptydir to download the plugins, both in openshift or not openshift scenario. Also I wish we can harden by default our securitycontext (with readonlyfs for instance) but in this period I do not have the amount of free time to add these changes, will try in the near future. In the mean time any PR is welcomed :)

BeastHook commented 2 months ago

That worked out great. I had to correct some plugins to higher versions because I use a newer LTS version of Jenkins. The container in the pod comes up to the startup of Jenkins and terminates without any further action after the log line "Jenkins is fully up and running".

jenkins-example-jenkins-master .log

brokenpip3 commented 2 months ago

Can you also attach the operator log?

brokenpip3 commented 2 months ago

Also kubernetes events if possible

BeastHook commented 2 months ago

Atm the Pod keeps getting a different problem: INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://ftp.halifax.rwth-aachen.de:443: Network is unreachable

Operator log: jenkins-operator-6cdbd9f879-52nkg-jenkins-operator.log

I dont have a eventrouter on the Openshift running. The events view shows: image

brokenpip3 commented 2 months ago

to me seems that kubelet that is terminating the pod

I dont have a eventrouter on the Openshift running.

you just need to run oc get ev -n <namespace or kubectl

also this part of the code is something I never touched, and will be hard to have a testing env, its seems kind of broken, I don't think we should handle the ocp route for the user, like we do with the ingress

2024-04-19T12:36:46.121Z DEBUG controller-jenkins Route API is available. Now creating route. {"cr": "msg"} 2024-04-19T12:36:46.132Z DEBUG controller-jenkins Jenkins Route is present {"cr": "msg"}

BeastHook commented 2 months ago

Good morning,

this is what i get. not really meaningful... Eventlog_jenkins.txt

Which parameter can I use to increase the log level?

The created route looks fine for me.

brokenpip3 commented 2 months ago

the issue is this one:

Jenkins master pod restarted by operator:; Jenkins pod security context has changed; Security context has changed

I don't know who is changing the security context of the pod. Right now it seems that we have:

I wish I can take a look but since I can't afford a ocp cluster it will be very hard to have a proper support for openshift 4.x.