planetscale / vitess-operator

Kubernetes Operator for Vitess
Apache License 2.0
304 stars 75 forks source link

Bug Report: Not able to list backups using kubectl get vitessbackups #626

Open frouioui opened 2 weeks ago

frouioui commented 2 weeks ago

Moving https://github.com/vitessio/vitess/issues/14766 here as it is related to the vitess-operator.

cc @GenLN as the original author.


Overview of the Issue

This is not working for 2 vitess cluster (only for one in separate namespace) on same K8S cluster.

Reproduction Steps

  1. Deploy operator and CRD into namespace A
  2. Deploy vitess cluster1 on namespace A
  3. Run full backups
  4. Run kubectl get vitessbackups into namespace A(it works)
  5. Update operator deployment and assign also to watch namespace B
  6. Deploy vitess cluster2 on namespace B
  7. Run full backups
  8. Run kubectl get vitessbackups into namespace B (empty for cluster2, even the backups are done and uploaded to gcs,s3 etc...)
  9. Run kubectl get vitessbackups into namespace A(list for cluster1 vitessbackups are shown)

Second test different K8S cluster:

  1. Deploy operator and CRD into namespace A1
  2. Deploy cluster11 into namespace A1
  3. Run full backups
  4. Run kubectl get vitessbackups into namespace A1 for cluster11(backup list shown)
  5. Deploy Operator deployment kind only into namespace B1
  6. Deploy cluster21 into namespace B1
  7. Run full backups
  8. Run kubectl get vitessbackups into namespace B1 for cluster 21 (empty list no backups shown)
  9. Run kubectl get vitessbackups into namespace A1 for cluster 11 (vitessbackups list shown)

Binary Version

Vitess v18.0.1
latest tag vitess operator

Operating System and Environment details

K8S cluster

Log Fragments

No response

frouioui commented 2 weeks ago

@GenLN, what is the reason behind using different namespaces? Also could you provide a more in depth reproduction, maybe a script with Kind.

GenLN commented 2 weeks ago

What I tried:

I encountered some issues with kubectl commands, like retrieving Vitess backups, etc.

I'm wondering if it's possible to run two Vitess clusters on the same Kubernetes cluster without them interfering with each other or causing issues with Kubernetes itself, such as with kubectl commands, etc.