kafka-ops / julie

A solution to help you build automation and gitops in your Apache Kafka deployments. The Kafka gitops!
MIT License
422 stars 115 forks source link

Fix Confluent Cloud ACL(s) support with JulieOps (can't no longer use AdminClient) #430

Closed purbon closed 2 years ago

purbon commented 2 years ago

Describe the bug Confluent Cloud latest releases break JulieOps ACLs management. Till now, JulieOps relied on using the ServiceAccount IDs and keeping a unified interface for ACLs management (Kafka own AdminClient).

This is no longer possible as with the latest release Confluent Cloud uses full SA name as part of their ACLs creation and no longer exposes Service Account IDs.

To Reproduce Steps to reproduce the behaviour: 1.- Provision a new (fresh) cluster in Confluent Cloud. 2.- Configure JulieOps to use the new Confluent Cloud cluster. 3.- Create a topology, be sure to Include ACLs management in it. Could be any topology as seen in the examples/ folder. 4.- Execute JulieOps

You will see empty ACL(s) if you list them with the Confluent Cloud CLI.

Expected behaviour

See all the expected ACLs created in the Confluent Cloud system.

purbon commented 2 years ago

Confluent Cloud API are finally generally available, ref https://docs.confluent.io/cloud/current/api.html#tag/Role-Bindings-(iamv2) this is the preferred API to be using for interacting with Confluent Cloud.

Possible solutions for this problem would include:

Important: CCloudCLi should be deprecated and removed as this functionality is no longer available and possible.

Amend https://github.com/kafka-ops/julie/blob/master/src/main/java/com/purbon/kafka/topology/roles/CCloudAclsProvider.java

In this class, we should:

Latest:

purbon commented 2 years ago

related #395