kafka-ops / julie

A solution to help you build automation and gitops in your Apache Kafka deployments. The Kafka gitops!
MIT License
418 stars 113 forks source link

RBAC - Schema registry subjects bindings are never deleted #513

Open damien-malescot opened 2 years ago

damien-malescot commented 2 years ago

Describe the bug When we remove subjects from a topoligy, bindings are not removed from RBAC

To Reproduce Create a topology file like this :

context: "DEV"
source: "example"
projects:
  - name: "demo"
    schemas:
      - principal: "Group:G_DEMO_READ"
        subjects:
          - "transactions"
    consumers:
      - principal: "Group:G_DEMO_READ"
    topics:
      - name: "personne.1"
        plan: "one-partition-compact"

Bindings are created :

        Principal        |     Role      | ResourceType |       Name       | PatternType
+------------------------+---------------+--------------+------------------+-------------+
  Group:G_DEMO_READ | DeveloperRead | Topic        | DEV.example.demo | PREFIXED
  Group:G_DEMO_READ | ResourceOwner | Group        | *                | LITERAL
        Principal        |     Role      | ResourceType |     Name     | PatternType
+------------------------+---------------+--------------+--------------+-------------+
  Group:G_DEMO_READ | ResourceOwner | Subject      | transactions | LITERAL

Remove all from topology and apply :

context: "DEV"
source: "example"
projects:
  - name: "demo"

Subject's bindings is still present :

-------------------------------------------------------------------------------
  Principal | Role | ResourceType | Name | PatternType
+-----------+------+--------------+------+-------------+
        Principal        |     Role      | ResourceType |     Name     | PatternType
+------------------------+---------------+--------------+--------------+-------------+
  Group:G_DEMO_READ | ResourceOwner | Subject      | transactions | LITERAL

Expected behavior Subjects's bindings must be removed when they are deleted from topology files.

You should use a property like for topics to recognize managed subjects : topology.subject.managed.prefixes";

purbon commented 2 years ago

Hi @damien-malescot, again thanks a lot for your help and reports, all very important.

I have done in #516 some validation of possible problems with the delete of subjects, sadly I'm not able to reproduce it yet. As you can see there, subjects gone, gone in my test.

Is anything specific you might have in your config file?

damien-malescot commented 2 years ago

Hi,

I'm on holidays, i will check that in 3 weeks. Or maybe @ludovic-boutros can check it before if he has time 😉

Thanks

ludovic-boutros commented 2 years ago

Hi @purbon , indeed it works with the master version. I will have to double check with @damien-malescot when he will be back, because I think this modification seems to fix the described issue: If not filtered, just apply updates.

Just to let you know, we have added a subject management prefix filter in our fork.

We would like to use the official version or at least something really similar in order to reduce our maintenance work. that's why we are creating all these issues in order to improve Julie. Hope this helps :)

purbon commented 2 years ago

Just to let you know, we have added a subject management prefix filter in our fork.

@ludovic-boutros, feel free to push a PR if that is ok for you! I'm very open to that kind of contribution! and I think that would help the project a lot.

Thanks a lot for all your efforts and help! it is honestly much appreciated.

ludovic-boutros commented 2 years ago

Hello @purbon, after some research with @damien-malescot , it seems that it is due to this parameter topology.state.cluster.enabled which is enabled on our platform. If I enable it in the test, it crashes.

One question, did you already try using random parameters for some tests or parametrized tests (Junit5) in order to tests a lot of parameter combinations ? (#534)