Open funky-eyes opened 1 month ago
I think, this won't work. There's an explicit prohibition in the code, which guards against the chicken-egg problem: how to bootstrap the remote metadata if they are on the remote storage themselves?
I think, this won't work. There's an explicit prohibition in the code, which guards against the chicken-egg problem: how to bootstrap the remote metadata if they are on the remote storage themselves?
This means that the local disk space must reserve the size of remote_log_metadata. For example, if the TTL of my other topics is six months, remote_log_metadata must be retained for more than six months.
@ivanyu Can __consumer_offsets topic be tiered?
What can we help you with?
Because I saw in Kafka documentation that it's desired for the persistence duration of this topic to be greater than the TTL of any other topic, I think this topic might occupy a significant amount of local disk space. Therefore, I would like to store it remotely, for example, in S3.
Where would you expect to find this information?