Redis Operator 0.15.1 does not work on the OpenShift Cluster Platform

[muhyid]

I am trying to install and run Redis Operator 0.15.1 through OpenShift Operator Hub on OpenShift Container Platform 4.11.49. I encountered four different problems:

1. Missing Cluster role for redisreplications and redissentinels resources: I solved the issue by adding these resources to the redis.redis.opstreelabs.in apiGroups in redis-operator.v0.15.1 Cluster Role
2. Missing Cluster role for redisclusters/status resource: I solved the issue by adding the resource to the redis.redis.opstreelabs.in apiGroups in redis-operator.v0.15.1 Cluster Role
3. Missing Cluster role for events resource: I solved the issue by adding the resource to the ' ' apiGroups in redis-operator.v0.15.1 Cluster Role
4. Can not create RedisSentinel, RedisReplication, RedisCluster, or Redis due to permission issue: RedisSentinel: Log from Pod /usr/bin/entrypoint-sentinel.sh: line 16: /etc/redis/sentinel.conf: Permission denied Starting sentinel service ..... 7:X 08 Oct 2023 11:53:46.169 # Sentinel config file /etc/redis/sentinel.conf is not writable: Permission denied. Exiting...

RedisReplication: Log from Pod Redis is running without password which is not recommended Setting up redis in standalone mode Running without TLS mode Starting redis service in standalone mode..... /usr/bin/entrypoint.sh: line 22: /etc/redis/redis.conf: Permission denied /usr/bin/entrypoint.sh: line 72: /etc/redis/redis.conf: Permission denied 8:C 08 Oct 2023 11:55:11.489 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 8:C 08 Oct 2023 11:55:11.489 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=8, just started 8:C 08 Oct 2023 11:55:11.489 # Configuration loaded 8:M 08 Oct 2023 11:55:11.489 monotonic clock: POSIX clock_gettime 8:M 08 Oct 2023 11:55:11.490 Running mode=standalone, port=6379. 8:M 08 Oct 2023 11:55:11.490 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 8:M 08 Oct 2023 11:55:11.490 # Server initialized 8:M 08 Oct 2023 11:55:11.490 * Ready to accept connections

$ redis-cli info replication
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:7fe526427251f3a107ba1ed45edf8858100e32ab
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

RedisCluster: Log from Pod Redis is running without password which is not recommended /usr/bin/entrypoint.sh: line 22: /etc/redis/redis.conf: Permission denied /usr/bin/entrypoint.sh: line 32: /etc/redis/redis.conf: Permission denied sed: /data/nodes.conf: No such file or directory Running without TLS mode Starting redis service in cluster mode..... /usr/bin/entrypoint.sh: line 72: /etc/redis/redis.conf: Permission denied 11:C 08 Oct 2023 12:01:19.246 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 11:C 08 Oct 2023 12:01:19.246 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=11, just started 11:C 08 Oct 2023 12:01:19.246 # Configuration loaded 11:M 08 Oct 2023 12:01:19.246 monotonic clock: POSIX clock_gettime 11:M 08 Oct 2023 12:01:19.247 Running mode=standalone, port=6379. 11:M 08 Oct 2023 12:01:19.247 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 11:M 08 Oct 2023 12:01:19.247 # Server initialized 11:M 08 Oct 2023 12:01:19.247 * Ready to accept connections

Redis: Event from StatefulSets create Pod redis-standalone-0 in StatefulSet redis-standalone failed error: pods "redis-standalone-0" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount

What version of redis operator are you using? Redis Operator 0.15.1 on OpenShift Cluster (trough Operator Hub)

oc logs redis-operator-5b9bdc9d48-fcxkf -n openshift-operators

{"level":"info","ts":1696768587.1184897,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"ns-rnd-redis-01","Request.PodDisruptionBudget.Name":"redis-sentinel-sentinel"} {"level":"info","ts":1696768587.2102492,"logger":"controllers.RedisSentinel","msg":"Will reconcile redis operator in again 10 seconds","Request.Namespace":"ns-rnd-redis-01","Request.Name":"redis-sentinel"} {"level":"info","ts":1696768597.2108173,"logger":"controllers.RedisSentinel","msg":"Reconciling opstree redis controller","Request.Namespace":"ns-rnd-redis-01","Request.Name":"redis-sentinel"} {"level":"error","ts":1696768597.2219589,"logger":"controller_redis","msg":"Failed to Execute Get Request","Request.RedisManager.Namespace":"ns-rnd-redis-01","Request.RedisManager.Name":"redis-sentinel","replication name":"redis-replication","namespace":"ns-rnd-redis-01","error":"redisreplications.redis.redis.opstreelabs.in \"redis-replication\" not found","stacktrace":"github.com/OT-CONTAINER-KIT/redis-operator/k8sutils.getSentinelEnvVariable\n\t/workspace/k8sutils/redis-sentinel.go:256\ngithub.com/OT-CONTAINER-KIT/redis-operator/k8sutils.generateRedisSentinelContainerParams\n\t/workspace/k8sutils/redis-sentinel.go:154\ngithub.com/OT-CONTAINER-KIT/redis-operator/k8sutils.RedisSentinelSTS.CreateRedisSentinelSetup\n\t/workspace/k8sutils/redis-sentinel.go:75\ngithub.com/OT-CONTAINER-KIT/redis-operator/k8sutils.CreateRedisSentinel\n\t/workspace/k8sutils/redis-sentinel.go:48\ngithub.com/OT-CONTAINER-KIT/redis-operator/controllers.(RedisSentinelReconciler).Reconcile\n\t/workspace/controllers/redissentinel_controller.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227"}

redis-operator version: 0.15.1

Does this issue reproduce with the latest release? Yes

What operating system and processor architecture are you using (kubectl version)? OpenShift Container Platform 4.11.49

kubectl version Output

$ kubectl version / oc version
Client Version: 4.13.8
Kustomize Version: v4.5.7
Server Version: 4.11.49
Kubernetes Version: v1.24.16+7aa7ea9

What did you do?

1. Install Redis Operator 0.15.1 provided by Opstree Solutions from OpenShift Operator Hub
2. Fix redisreplications, redissentinels, redisclusters/status, and events resources issues
3. Try to create RedisSentinel, RedisReplication, RedisCluster, or Redis from installed operators section for differenet namespaces

What did you expect to see? First of all RedisSentinel but not only this, all options should work without any errors.

What did you see instead? Cluster Operator and Redis Workloads didn't run

[shubham-cmyk]

@muhyid Can you provide me the manifest that you have applied?

[muhyid]

Hi @shubham-cmyk

It is from OpenShift Operator Hub.

This is your Cluster Opeator and Subnscription YAMLs:

ocp-redis-cluster-operator.txt

ocp-redis-sentinel.txt

[usevalad-prus]

Hello, we have the same issue with RedisCluster on OCP -> Permission Denied. We are also not able to set securityContext in RedisCluster CRD.

[cdmikechen]

You need to add a cluster role binding to your redis service account. Like this:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: {{ .Release.Name }}-redis-rolebinding
  namespace: {{ .Release.Namespace }}
subjects:
  - kind: ServiceAccount
    name: redis
    namespace: {{ .Release.Namespace }}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: {{ include "runAnyRole" . }}
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: {{ include "runAnyRole" . }}
  namespace: {{ .Release.Namespace }}
rules:
  - verbs:
      - use
    apiGroups:
      - security.openshift.io
    resources:
      - securitycontextconstraints
    resourceNames:
      - anyuid
      - privileged

[muhyid]

Hi @cdmikechen This is fo my issue or @usevalad-prus issue?

[LennertMertens]

I'm having similar issues. Any updates on this yet?

[vincentmele]

This bug report does a great job of collecting all of the various challenges with using this operator with OpenShift and OKD.

There's no reason that redis needs to run as root inside these containers. There's no reason it needs the full permissions described in #416.

[gazdagandras]

Hi!

The documentation is not up to date. With redis-opertator version 0.15.1 use v7.2.3 redis and redis-sentinel images! They contain updates to run on OKD /OpenShift clusters.