search

oracle / oracle-database-operator

The Oracle Database Operator for Kubernetes (a.k.a. OraOperator) helps developers, DBAs, DevOps and GitOps teams reduce the time and complexity of deploying and managing Oracle Databases. It eliminates the dependency on a human operator or administrator for the majority of database operations.
Universal Permissive License v1.0
131 stars 43 forks source link

DB can not be created on a normal namespace in OpenShift (security!) #100

Open rbaumgar opened 1 month ago

rbaumgar commented 1 month ago

When I try to deploy the xe-sample to a namespace like "oracle". Operator is not able to create deployment/pod.

runAsUser=54321 is not allowed by default.

2024-05-17T13:12:15Z    INFO    singleinstancedatabase-resource default {"name": "xedb"}
2024-05-17T13:12:15Z    INFO    singleinstancedatabase-resource validate create {"name": "xedb"}
2024-05-17T13:12:15Z    INFO    controllers.database.SingleInstanceDatabase Reconcile requested
2024-05-17T13:12:15Z    INFO    singleinstancedatabase-resource default {"name": "xedb"}
2024-05-17T13:12:16Z    INFO    singleinstancedatabase-resource validate update {"name": "xedb"}
2024-05-17T13:12:16Z    INFO    singleinstancedatabase-resource validate create {"name": "xedb"}
2024-05-17T13:12:16Z    INFO    controllers.database.SingleInstanceDatabase Entering reconcile validation
2024-05-17T13:12:16Z    INFO    controllers.database.SingleInstanceDatabase Completed reconcile validation
2024-05-17T13:12:16Z    INFO    controllers.database.SingleInstanceDatabase Creating a new PVC  {"createPVC Datafiles-Vol": {"name":"xedb","namespace":"oracle"}, "PVC.Namespace": "oracle", "PVC.Name": "xedb"}
2024-05-17T13:12:16Z    INFO    No xedb Pod is Ready    {"controller": "singleinstancedatabase", "controllerGroup": "database.oracle.com", "controllerKind": "SingleInstanceDatabase", "SingleInstanceDatabase": {"name":"xedb","namespace":"oracle"}, "namespace": "oracle", "name": "xedb", "reconcileID": "3c0e7686-7d38-4654-8ef4-4bbc76e8fbd7", "FindPods": {"name":"xedb","namespace":"oracle"}}
2024-05-17T13:12:16Z    INFO    xedb Pods Available ( Other Than Ready Pod )    {"controller": "singleinstancedatabase", "controllerGroup": "database.oracle.com", "controllerKind": "SingleInstanceDatabase", "SingleInstanceDatabase": {"name":"xedb","namespace":"oracle"}, "namespace": "oracle", "name": "xedb", "reconcileID": "3c0e7686-7d38-4654-8ef4-4bbc76e8fbd7", "FindPods": {"name":"xedb","namespace":"oracle"}, " Names :": []}
2024-05-17T13:12:16Z    INFO    Total No Of xedb PODS   {"controller": "singleinstancedatabase", "controllerGroup": "database.oracle.com", "controllerKind": "SingleInstanceDatabase", "SingleInstanceDatabase": {"name":"xedb","namespace":"oracle"}, "namespace": "oracle", "name": "xedb", "reconcileID": "3c0e7686-7d38-4654-8ef4-4bbc76e8fbd7", "FindPods": {"name":"xedb","namespace":"oracle"}, "Count": 0}
2024-05-17T13:12:16Z    INFO    controllers.database.SingleInstanceDatabase Replica Info    {"createPods": {"name":"xedb","namespace":"oracle"}, "Found": 0, "Required": 1}
2024-05-17T13:12:16Z    INFO    controllers.database.SingleInstanceDatabase Creating a new xedb POD {"createPods": {"name":"xedb","namespace":"oracle"}, "POD.Namespace": "oracle", "POD.Name": "xedb-5qt1e"}
2024-05-17T13:12:16Z    ERROR   controllers.database.SingleInstanceDatabase Failed to create new xedb POD   {"createPods": {"name":"xedb","namespace":"oracle"}, "pod.Namespace": "oracle", "POD.Name": "xedb-5qt1e", "error": "pods \"xedb-5qt1e\" is forbidden: unable to validate against any security context constraint: [provider \"anyuid\": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{54321}: 54321 is not an allowed group, provider restricted-v2: .initContainers[0].runAsUser: Invalid value: 54321: must be in the ranges: [1000700000, 1000709999], provider restricted-v2: .containers[0].runAsUser: Invalid value: 54321: must be in the ranges: [1000700000, 1000709999], provider restricted-v2: .containers[0].capabilities.add: Invalid value: \"SYS_NICE\": capability may not be added, provider \"restricted\": Forbidden: not usable by user or serviceaccount, provider \"nonroot-v2\": Forbidden: not usable by user or serviceaccount, provider \"nonroot\": Forbidden: not usable by user or serviceaccount, provider \"hostmount-anyuid\": Forbidden: not usable by user or serviceaccount, provider \"machine-api-termination-handler\": Forbidden: not usable by user or serviceaccount, provider \"hostnetwork-v2\": Forbidden: not usable by user or serviceaccount, provider \"hostnetwork\": Forbidden: not usable by user or serviceaccount, provider \"hostaccess\": Forbidden: not usable by user or serviceaccount, provider \"hostpath-provisioner\": Forbidden: not usable by user or serviceaccount, provider \"privileged\": Forbidden: not usable by user or serviceaccount]"}
github.com/oracle/oracle-database-operator/controllers/database.(*SingleInstanceDatabaseReconciler).createPods
    /workspace/controllers/database/singleinstancedatabase_controller.go:2151
github.com/oracle/oracle-database-operator/controllers/database.(*SingleInstanceDatabaseReconciler).createOrReplacePods
    /workspace/controllers/database/singleinstancedatabase_controller.go:1915
github.com/oracle/oracle-database-operator/controllers/database.(*SingleInstanceDatabaseReconciler).Reconcile
    /workspace/controllers/database/singleinstancedatabase_controller.go:189
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
    /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
    /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:227
rbaumgar commented 1 month ago

How can I remove runAsUser property?

yunus-qureshi commented 1 month ago

@rbaumgar for openshift envs, you must apply this yaml

https://github.com/oracle/oracle-database-operator/blob/main/config/samples/sidb/openshift_rbac.yaml

and specify the service account name "sidb-sa" in the SIDB yaml

rbaumgar commented 1 month ago

This might be a workaround, but is never a solution. Every normal pod has to run with an arbitrary uid. Sorry, a database is a normal pod and does not require special security requirements. You will find much more information on this and several other links https://developers.redhat.com/articles/2021/11/11/best-practices-building-images-pass-red-hat-container-certification#

yunus-qureshi commented 1 month ago

Agreed. The latest v1.1.0 has an attribute called setWritePermissions. Set it to false

https://github.com/oracle/oracle-database-operator/blob/main/config/samples/sidb/singleinstancedatabase.yaml

rbaumgar commented 1 month ago

does not work on singleinstancedatabase_express.yaml

Invalid value: []int64{54321}: 54321 is not an allowed group, provider restricted-v2: .initContainers[0].runAsUser: Invalid value: 54321: must be in the ranges: ... 

spec:
  adminPassword:
    keepSecret: true
    secretKey: oracle_pwd
    secretName: xedb-admin-secret
  createAs: primary
  edition: express
  image:
    prebuiltDB: true
    pullFrom: 'container-registry.oracle.com/database/express:latest'
  pdbName: XEPDB1
  persistence:
    accessMode: ReadWriteOnce
    setWritePermissions: false
    size: 50Gi
    storageClass: oci-bv
  replicas: 1
  sid: XE
yunus-qureshi commented 1 month ago

@rbaumgar also set the attribute prebuiltDB to false

andbos commented 1 month ago

Hi,

When I try to apply openshift_rbac.yaml I get the following error:

$ oc apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/config/samples/sidb/openshift_rbac.yaml
serviceaccount/sidb-sa created
role.rbac.authorization.k8s.io/use-sidb-scc created
rolebinding.rbac.authorization.k8s.io/use-sidb-scc created
error: resource mapping not found for name: "sidb-scc" namespace: "default" from "https://raw.githubusercontent.com/oracle/oracle-database-operator/main/config/samples/sidb/openshift_rbac.yaml": no matches for kind "SecurityContextConstraints" in version "v1"
ensure CRDs are installed first

Installation of the operator went fine:

$ oc apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml
namespace/oracle-database-operator-system created
customresourcedefinition.apiextensions.k8s.io/autonomouscontainerdatabases.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/autonomousdatabasebackups.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/autonomousdatabaserestores.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/autonomousdatabases.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/cdbs.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/databaseobservers.observability.oracle.com created
customresourcedefinition.apiextensions.k8s.io/dataguardbrokers.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/dbcssystems.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/oraclerestdataservices.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/pdbs.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/shardingdatabases.database.oracle.com created
customresourcedefinition.apiextensions.k8s.io/singleinstancedatabases.database.oracle.com created
role.rbac.authorization.k8s.io/oracle-database-operator-leader-election-role created
clusterrole.rbac.authorization.k8s.io/oracle-database-operator-manager-role created
clusterrole.rbac.authorization.k8s.io/oracle-database-operator-metrics-reader created
clusterrole.rbac.authorization.k8s.io/oracle-database-operator-oracle-database-operator-proxy-role created
rolebinding.rbac.authorization.k8s.io/oracle-database-operator-oracle-database-operator-leader-election-rolebinding created
rolebinding.rbac.authorization.k8s.io/oracle-database-operator-oracle-database-operator-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/oracle-database-operator-proxy-rolebinding created
service/oracle-database-operator-controller-manager-metrics-service created
service/oracle-database-operator-webhook-service created
certificate.cert-manager.io/oracle-database-operator-serving-cert created
issuer.cert-manager.io/oracle-database-operator-selfsigned-issuer created
mutatingwebhookconfiguration.admissionregistration.k8s.io/oracle-database-operator-mutating-webhook-configuration created
validatingwebhookconfiguration.admissionregistration.k8s.io/oracle-database-operator-validating-webhook-configuration created
deployment.apps/oracle-database-operator-controller-manager created

$ oc -n oracle-database-operator-system get pods
NAME                                                           READY   STATUS    RESTARTS   AGE
oracle-database-operator-controller-manager-7f84b7dc4b-994lm   1/1     Running   0          18s
oracle-database-operator-controller-manager-7f84b7dc4b-t5j7r   1/1     Running   0          18s
oracle-database-operator-controller-manager-7f84b7dc4b-twf7d   1/1     Running   0          18s
rbaumgar commented 1 month ago

@andbos this works only on OpenShift. Openshift has an SCC object:

$ oc get crd securitycontextconstraints.security.openshift.io -o yaml|grep storedVersion -A2
  storedVersions:
  - v1
andbos commented 1 month ago

Yes, started testing in OpenShift.

$ oc version
Client Version: 4.14.11
Kustomize Version: v5.0.1
Server Version: 4.14.12
Kubernetes Version: v1.27.10+28ed2d7

The instance was installed properly anyway...

$ oc -n default get singleinstancedatabase
NAME            EDITION      STATUS    ROLE      VERSION      CONNECT STR                                                                            TCPS CONNECT STR   OEM EXPRESS URL
sinchdb11rhos   Enterprise   Healthy   PRIMARY   21.3.0.0.0   605682735.eu-west-1.elb.amazonaws.com:1521/RHOSDB11   Unavailable        https://605682735.eu-west-1.elb.amazonaws.com:5500/em

No errors in the operator logs.

rbaumgar commented 1 month ago

Oh, I see. The SCC is completely wrong formated and the api version is wrong. :-(