redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.22k stars 564 forks source link

cloud_storage: API to disable and purge tiered storage #7918

Open jcsp opened 1 year ago

jcsp commented 1 year ago

Setting redpanda.remote.write=false stops the upload process, but does not clear anything up.

If a topic is having issues with tiered storage, or simply was enabled mistakenly, we should provide a way to disabled tiered storage that entirely cleans up & leaves the topic in the same state it would have been if tiered storage had never been used to begin with:

JIRA Link: CORE-1122

mattschumpert commented 1 year ago

@jcsp we have now seen multiple beta testing users desire this, in the course of toggling on and off TS for existing topics (testing) and in the course of migrating from one object store type/location to another. There are workaround but have been messier and time consuming. So +1 to this