search

devshawn / kafka-gitops

🚀Manage Apache Kafka topics and generate ACLs through a desired state file.
https://devshawn.github.io/kafka-gitops
Apache License 2.0
321 stars 71 forks source link

Feature Request: Support Topic Whitelisting #90

Open deepbrook opened 2 years ago

deepbrook commented 2 years ago

Currently, only topic black-listing is supported, which works great for centralized ACL management.

However, it makes decentralized topic management difficult. We have multiple teams managing their own kafka topics on the same cluster. They currently need to blacklist each of the other teams' prefix. Whenever we onboard new teams, the blacklist needs updating - a very error-prone process, because, you know, people.

How should this work?

It would be an exact inversion of the current blacklisting:

settings:
    whitelist:
      prefixed: 
        - only.my.topics.

..would only manage topics with the above prefix - this also means that any topics declared in the state file NOT matching this prefix are ignored (or perhaps throw an error).

The two mechanisms would be expected to be mutually exclusive.

settings:
    whitelist: {...}
    blacklist: {...}  # Specifying both types raises a validation error
joschi commented 2 years ago

FYI: I've added this functionality to my fork of kafka-gitops.

➡️ https://github.com/joschi/kafka-gitops/releases/tag/0.3.0

Please note the renaming of blacklist to exclude and the corresponding include setting.