michelin / ns4kafka

Ns4Kafka brings namespaces on top of Kafka brokers, Kafka Connect and Schema Registry.
Apache License 2.0
73 stars 12 forks source link

Handle Confluent Cloud tags #282

Closed loicgreffier closed 11 months ago

loicgreffier commented 1 year ago

Ns4Kafka should provide the ability to tag resources (topics and schemas) with Confluent Cloud tags: https://docs.confluent.io/cloud/current/api.html#tag/Types-(v1)/operation/createTagDefs.

Possible implementation

Users

Users should be able to tag their resources (schemas and topics) when deployed on Confluent Cloud only. The main solution is adding a new field in the resource metadata that contains all the tags split by a comma:

---
apiVersion: v1
kind: Topic
metadata:
  name: myPrefix.topic
  tags: tag1,tag2,tag3
spec:
...

The list of possible tags for a namespace should be defined by admins, and validated at namespace level:

---
apiVersion: v1
kind: Namespace
metadata:
  name: myNamespace
  cluster: myCluster
  labels:
    contacts: namespace.owner@example.com
spec:
  topicValidator:
  connectValidator:
  tagValidator:
      tags:
        validation-type: ValidString
        validStrings:
          - tag1
          - tag2
          - tag3

A dedicated executor for tags might need to be implemented to handle both schema tags and topic tags.

Admin

Tags might need to be declared in the Confluent Cloud before getting attached to a resource. In this case, 3 possible solutions:

---
apiVersion: v1
kind: Tag
metadata:
  name: tag1
spec:
  description: This is a wondeful tag

This option may not make sens in Ns4Kafka as it is unrelated to namespaces at this point.

loicgreffier commented 1 year ago

The validation at namespace level could be omitted. All namespaces should have the right to use all the CCloud tags.

A remote validation against Schema Registry API during topic/schema deployment to check that the defined tags are compliant should be performed

loicgreffier commented 11 months ago

https://github.com/michelin/ns4kafka/pull/321