search

redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.47k stars 579 forks source link

suggestion: K8s operator should put files in subdirectory #1740

Open shribe opened 3 years ago

shribe commented 3 years ago

Not every PVC mounted is going to be dynamically-created by a CSI. Sometimes we need to look at backing disk to see what's there. Sometimes multiple pods will have access to the same volume, for instance local storage (of course this can be managed by the user, by creating the subdirectory and pointing the PV to it, but it would be nicer to not have to worry about it).

I suggest creating a directory named , or redpanda-, or redpanda-- if there's an appropriate identifier available.

Just as 2 data points: the CrunchData operator does this, and it is quite nice for managing storage (snapshots, moving old data aside temporarily, copying backup into place for restoration), on the other hand CockroachDB spews data files all over the top level. (At least you only put a few files there and put data in a subdirectory.)

JIRA Link: CORE-667

emaxerrno commented 3 years ago

@shribe I love these issues! thanks for helping us make redpanda better.

This is a great idea too