search

Aiven-Open / tiered-storage-for-apache-kafka

RemoteStorageManager for Apache Kafka® Tiered Storage
Apache License 2.0
91 stars 19 forks source link

fix: add empty indexes to builder #469

Closed jeqo closed 9 months ago

jeqo commented 9 months ago

By ignoring empty indexes--that is a valid case when segment sizes are too small; even if not recommended--the requirement to have at least 4 indexes when uploaded was not met.

This fix improves the logging to show what indexes are included therefore know which one is missing; and includes a fix to add empty indexes to the builder.

See commits for more details. It includes moving some RSM tests from IT to unit tests.

ivanyu commented 9 months ago

A question about refactor: further moving integration to unit tests for RSM: what's the purpose of this move? AFAICS, we still use the same FileSystemStorage, so it's still a running real RSM with real backend. Or I'm missing some other change?

ivanyu commented 9 months ago

Apart from that question, LGTM

jeqo commented 9 months ago

AFAICS, we still use the same FileSystemStorage, so it's still a running real RSM with real backend.

Oh, I was under the impression that we would keep only long-running tests on integration tests. Let's move them back, with the additional changes.

jeqo commented 9 months ago

@ivanyu RSMTest changes are in place now, have a look. Thanks!