search

redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.41k stars 577 forks source link

Topic config `redpanda.remote.readreplica` is never included in metadata #8021

Open patrickangeles opened 1 year ago

patrickangeles commented 1 year ago

Version & Environment

22.3

What went wrong?

When creating a read replica topic, we set a parameter redpanda.remote.readreplica=bucketname. This parameter is never returned in topic metadata queries, which plays badly with tools like Terraform which does a diff between current and desired state.

JIRA Link: CORE-1131

patrickangeles commented 1 year ago

Related https://github.com/redpanda-data/redpanda/discussions/7859

mattschumpert commented 1 year ago

To be specific, @patrickangeles confirmed via rpk that this property was not showing up when you do

rpk topic describe -a foo

mattschumpert commented 1 year ago

@twmb any ideas here? @mmaslankaprv should't every property always be part of the topic metadata?

daisukebe commented 1 year ago

Just a side node: even we decide to not expose the read replica related flags via describe_configs, it'd be greatly helpful if there's a way to figure out all current topic properties, e.g. say via an admin api.

vuldin commented 1 year ago

Related to this issue is that redpanda.remote.read and redpanda.remote.write do appear, and for a read replica topic their values are both true. This is problematic though, since it doesn't make sense for a read replica topic to have remote.write set to true. Not only do users not see readreplica, but they also see incorrect values.

KiaRaghavan commented 1 year ago

thanks @vuldin for documenting the issues with the read replica config values in topic describe