Aiven-Open / tiered-storage-for-apache-kafka

RemoteStorageManager for Apache Kafka® Tiered Storage
Apache License 2.0
91 stars 19 forks source link

feat: extend object deleter to delete multiple keys at once when supported by back-end #490

Closed jeqo closed 7 months ago

jeqo commented 8 months ago

Some Storage back-ends have the ability to delete multiple objects at once. To allow them to use this optimization, the ObjectDeleter interface is extended to allow passing multiple object keys at once.

Fix #319

PS. Was considering GCS initially, but given that FakeGcs server doesn't seem to properly handle the delete(List of BlobId) -- at least metrics where not processed -- and implementation seems to be syntactic sugar; then I decided to postpone it and only focused on S3.