Feature: apply changes following plan order

[gquintana]

When running

  topics:
    - name: topic1
      partitions: 3
      replication_factor: 2
    - name: topic2
      partitions: 3
      replication_factor: 2

The output is

TASK [CREATE] Create a new topic topic2 (partitions=3, replicas=2) - OK ****************************
TASK [CREATE] Create a new topic topic1 (partitions=3, replicas=2) - OK ****************************

Topics are created following a random order, it makes it hard

• to check what's being done (or about to be done when dry run mode)
• to debug the plan content

I'd like the plan ordered to be respected

[gquintana]

The source problem is that all changes are stored as Maps.

Trying to implement this feature, I noticed that computing changeset to apply was done differently in ACLs and Topics. I tried to refactor the TopicChange/AclChange to share same logic, this is how and why I created ChangeComputer My current code is here https://github.com/gquintana/kafka-specs/tree/feat/ordered-changes/src/main/java/io/streamthoughts/kafka/specs/change It's far from being finished (it doesn't compile). I am having hard times bridging this with *Operation, I fear it's a dead end, and I am not sure whether this change is a good idea or not.

What do you think? Should I dig deaper?

[fhussonnois]

Currently, the Change API is not very well designed...no doubt about it! But, the ChangeComputer class that you introduce seems to be a good idea. I've recently added support for quotas and changes are computed mostly like for topics.

Also, changes for Topics/Quotas should accept some flags to indicate if deletions should be computed (like for ACLs). This is related to this issue : https://github.com/streamthoughts/kafka-specs/issues/45

Unfortunately, I'm not sure if, in the end, it's possible to keep the order of execution, as changes are applied in batches and by type (i.e. delete, create and modify).