Support persistent storage

scylladb / scylla-operator

The Kubernetes Operator for ScyllaDB

https://operator.docs.scylladb.com/

Apache License 2.0

324 stars 159 forks source link

Support persistent storage #686

Open tnozicka opened 2 years ago

tnozicka commented 2 years ago

Is this a bug report or feature request?

Feature Request

What should the feature do:

Support PV backed by persistent storage
Add e2e variant for persistent storage

What is use case behind this feature: Some users don't always need the performance, e.g. for small or test clusters. Dedicated SSDs cost a lot and it requires additional setup to create the local static provisioner.

tnozicka commented 2 years ago

We need to document that persistent storage won't work with hostNetworking because the pods won't stick to the nodes but they use their IPs for identity. This works only with local storage because PVs has a node affinity and pods will stick to it.

tnozicka commented 2 years ago

Some noted that it may work if we use internal ips for membership, so it may just need verification.

tnozicka commented 2 years ago

We are using the ClusterIP for identity so even with hostNetworking we wouldn't care about the Pod IP changing. I guess we just need some testing coverage.

scylla-operator-bot[bot] commented 2 weeks ago

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 30d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out

/lifecycle stale

tnozicka commented 2 weeks ago

/triage accepted let's add a test that chooses standard-rwo on GKE and runs our conformance suite

zimnx commented 2 weeks ago

let's add a test that chooses standard-rwo on GKE and runs our conformance suite

*-clusterip variant already runs on persistent disks (pd-ssd)

tnozicka commented 2 weeks ago

To my knowledge none of our tests uses storageclass other then scylladb-local-xfs.

Are you talking about the disks used the VMs?

zimnx commented 2 weeks ago

I'm talking about disks used by scylladb-local-xfs. On parallel and serial jobs, scylladb-local-xfs is backed by localSSD, where on parallel-clusterip is backed by pd-ssd.

rzetelskik commented 2 weeks ago

I'm talking about disks used by scylladb-local-xfs. On parallel and serial jobs, scylladb-local-xfs is backed by localSSD, where on parallel-clusterip is backed by pd-ssd.

The point is it still uses our driver in that case. Having said that, what's the value in testing against a different storage class @tnozicka? We wouldn't encourage anyone to run with standard-rwo anyway, so why cover it?

tnozicka commented 1 week ago

While we don't recommend it for performance reasons, people still use it. It's fine for instances that don't care about performance. The PVC lifecycle is a bit different and we should make sure we don't break conformance suite, nothing more.