Jenkins Controller and Build Pods in different namespaces

[dee-kryvenko]

Need to be able to have build pods and jenkins controller in separate namespaces. Doesn't seem to be currently possible, which pose the risk of user-defined jobs to be able to construct such a pod that uses SA from controller so it can read secrets, amend controller pod etc. There are four separate roles in a typical shared k8s setup - cluster admins, operator admins, controller admins and jobs admins.

[devdattakulkarni]

+1 From multi-tenancy perspective also this is an important requirement - that the Operator is able to support creating Jenkins instances in different namespaces.

[thecooldrop]

Hi @dee-kryvenko, could you provide more details as to why this is not possible? I am unable to come to the same conclusion as you

[dee-kryvenko]

@thecooldrop because https://github.com/jenkinsci/kubernetes-operator/blob/v0.7.0/pkg/configuration/base/resources/base_configuration_configmap.go#L207 is clearly using jenkins.ObjectMeta.Namespace without any ability for the user to override it.

[dashashutosh24]

I was able to resolve this by creating a copy of the role and rolebinding from the namespace in which jenkins master resides in the namespace where I want the slave pods to run. In the rolebinding, I granted the master service account in the master namespace access to the role in the slave agent namespace.

[brokenpip3]

I was able to resolve this by creating a copy of the role and rolebinding from the namespace in which jenkins master resides in the namespace where I want the slave pods to run. In the rolebinding, I granted the master service account in the master namespace access to the role in the slave agent namespace.

so you are saying that in your tests it's just a rbac issue because the jenkins operator does not have the grants to create pods, cm, secrets etc in the others namespace?

I never tried this, maybe it's something we can fix easily

[dashashutosh24]

Yes it seems it was only a permissions issue. This could be the rolebinding in the slave build agents' namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: jenkins-operator-jenkins
  namespace: jenkins-slaves
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins-operator-jenkins
subjects:
- kind: ServiceAccount
  name: jenkins-operator-jenkins
  namespace: jenkins-operator

The role is an exact copy of the role in master's namespace with the same permissions and the rolebinding binds it with the serviceaccount used by master in it's own namespace.