search

rabbitmq / cluster-operator

RabbitMQ Cluster Kubernetes Operator
https://www.rabbitmq.com/kubernetes/operator/operator-overview.html
Mozilla Public License 2.0
872 stars 273 forks source link

Incompatible with kubernetes 1.27 #1351

Closed Snazzie closed 1 year ago

Snazzie commented 1 year ago

Describe the bug

this package seems to be not compatible with kubernetes 1.27. Originally reported in bitnami/rabbitmq-cluster-operator then was suggested to raise issue here. Might have something to do with golang version? worked in 1.26

To Reproduce

Steps to reproduce the behavior:

  1. have kubernetes 1.27 cluster
  2. deploy rabbitmq-cluster-operator with configurations and values
  3. rabbitmq-messaging-topology-operator and rabbitmq-rabbitmq-messaging-topology-operator pod fails

Include any YAML or manifest necessary to reproduce the problem.

Expected behavior pods are healthy

Screenshots

If applicable, add screenshots to help explain your problem.

Version and environment information

Additional context

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1277ad0]

goroutine 1 [running]:
k8s.io/client-go/discovery.convertAPIResource(...)
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/aggregated_discovery.go:114
k8s.io/client-go/discovery.convertAPIGroup({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc000121458, 0x15}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/aggregated_discovery.go:95 +0x7d0
k8s.io/client-go/discovery.SplitGroupsAndResources({{{0xc0001200f0, 0x15}, {0xc00050c3e0, 0x1b}}, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/aggregated_discovery.go:49 +0x318
k8s.io/client-go/discovery.(*DiscoveryClient).downloadAPIs(0x98?)
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/discovery_client.go:328 +0x3de
k8s.io/client-go/discovery.(*DiscoveryClient).GroupsAndMaybeResources(0x0?)
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/discovery_client.go:203 +0x65
k8s.io/client-go/discovery.ServerGroupsAndResources({0x1b92498, 0xc00034de90})
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/discovery_client.go:413 +0x59
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources.func1()
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/discovery_client.go:376 +0x25
k8s.io/client-go/discovery.withRetries(0x2, 0xc000298d68)
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/discovery_client.go:651 +0x72
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources(0x0?)
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/discovery/discovery_client.go:375 +0x3a
k8s.io/client-go/restmapper.GetAPIGroupResources({0x1b92498?, 0xc00034de90?})
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/k8s.io/client-go@v0.26.3/restmapper/discovery.go:148 +0x42
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper.func1()
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/client/apiutil/dynamicrestmapper.go:94 +0x25
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).setStaticMapper(...)
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/client/apiutil/dynamicrestmapper.go:130
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper(0xc000492240?, {0x0, 0x0, 0xd93cc79e2c40e701?})
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/client/apiutil/dynamicrestmapper.go:110 +0x182
sigs.k8s.io/controller-runtime/pkg/cluster.setOptionsDefaults.func1(0xc0001f4620?)
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/cluster/cluster.go:217 +0x25
sigs.k8s.io/controller-runtime/pkg/cluster.New(0xc00029cb40, {0xc00068f708, 0x1, 0x0?})
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/cluster/cluster.go:159 +0x18d
sigs.k8s.io/controller-runtime/pkg/manager.New(_, {0xc0001f4620, 0x0, 0x0, {{0x1b8f2b8, 0xc000409500}, 0x0}, 0x1, {0x0, 0x0}, ...})
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/manager/manager.go:351 +0xf9
main.main()
    /bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.10.3/src/github.com/rabbitmq/rmq-messaging-topology-operator/main.go:133 +0x8d5
panic: runtime error: invalid memory address or nil pointer dereference

[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12a6750]

goroutine 1 [running]:

k8s.io/client-go/discovery.convertAPIResource(...)

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/aggregated_discovery.go:88

k8s.io/client-go/discovery.convertAPIGroup({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc0004f7200, 0x15}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/aggregated_discovery.go:69 +0x5f0

k8s.io/client-go/discovery.SplitGroupsAndResources({{{0xc0004f6360, 0x15}, {0xc00004cc40, 0x1b}}, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/aggregated_discovery.go:35 +0x2f8

k8s.io/client-go/discovery.(*DiscoveryClient).downloadAPIs(0x1a2ce30?)

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:310 +0x47c

k8s.io/client-go/discovery.(*DiscoveryClient).GroupsAndMaybeResources(0xc0001a6e60?)

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:198 +0x5c

k8s.io/client-go/discovery.ServerGroupsAndResources({0x1d1b260, 0xc000155140})

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:392 +0x59

k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources.func1()

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:356 +0x25

k8s.io/client-go/discovery.withRetries(0x2, 0xc0000dd048)

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:621 +0x72

k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources(0x0?)

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:355 +0x3a

k8s.io/client-go/restmapper.GetAPIGroupResources({0x1d1b260?, 0xc000155140?})

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/k8s.io/client-go@v0.26.1/restmapper/discovery.go:148 +0x42

sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper.func1()

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/client/apiutil/dynamicrestmapper.go:94 +0x25

sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).setStaticMapper(...)

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/client/apiutil/dynamicrestmapper.go:130

sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper(0xc00013c120?, {0x0, 0x0, 0xd93cc79e2c40e701?})

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/client/apiutil/dynamicrestmapper.go:110 +0x182

sigs.k8s.io/controller-runtime/pkg/cluster.setOptionsDefaults.func1(0xc0000f88c0?)

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/cluster/cluster.go:217 +0x25

sigs.k8s.io/controller-runtime/pkg/cluster.New(0xc0003d2000, {0xc0005139f0, 0x1, 0x0?})

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/cluster/cluster.go:159 +0x18d

sigs.k8s.io/controller-runtime/pkg/manager.New(_, {0xc0000f88c0, 0x0, 0x0, {{0x1d167c0, 0xc00061e7c0}, 0x0}, 0x1, {0x0, 0x0}, ...})

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/manager/manager.go:351 +0xf9

main.main()

    /bitnami/blacksmith-sandox/rabbitmq-cluster-operator-2.2.0/src/github.com/rabbitmq/cluster-operator/main.go:124 +0xb5a
DanielePalaia commented 1 year ago

Hi @Snazzie we usually use GKE for our tests but still it seems like it doesn't support Kubernetes 1.27 yet. I tested anyway with minikube and rancher (rancher supports 1.27.1) and there installing both with the manifests and with helm (https://bitnami.com/stack/rabbitmq-cluster-operator/helm) seems working fine. Which K8s are you using? Are you maybe modifying some values? Did you try also installing from the main manifest? https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml

Snazzie commented 1 year ago

Hi @DanielePalaia, i used kurl.sh to create my cluster on a fresh ubuntu vm. Prior to this install, my cluster was on 1.26, charts were installed via argocd and didnt have any issues.

In this new cluster, every chart apart from this one installed ok.

I now tried installing the chart in minikube running 1.27.1 via cli and its healthy...