Ability to move data between JBOD disks using Cruise Control

[ShubhamRwt]

Type of change

This PR adds the ability to move data between two JBOD disks using Cruise Control. To do the same, we have introduced a new mode called remove-disks. You can use it like this:

yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaRebalance
metadata:
  name: my-rebalance
  labels:
    strimzi.io/cluster: my-cluster
spec:
  # setting the mode as `remove-disks` to move data between the JBOD disks
  mode: remove-disks
  # providing the list of brokers, and the corresponding volumes from which you want to move the replicas
  moveReplicasOffVolumes:
    - brokerId: 0
      volumeIds: [1, 2]
    - brokerId: 2
      volumeIds: [1]
# ...

• Bugfix
• Enhancement / new feature
• Refactoring
• Documentation

Description

Please describe your pull request

Checklist

Please go through this checklist and make sure all applicable tasks have been done

• [ ] Write tests
• [ ] Make sure all tests pass
• [ ] Update documentation
• [ ] Check RBAC rights for Kubernetes / OpenShift roles
• [ ] Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
• [ ] Reference relevant issue(s) and close them after merging
• [ ] Update CHANGELOG.md
• [ ] Supply screenshots for visual changes, such as Grafana dashboards

[scholzj]

/azp run regression

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[ppatierno]

@ShubhamRwt I would rebase against main, because a refactoring around getting task status of rebalance was just merged. It would be better for you to have the code up to date while addressing comments on this PR. Thanks!

[ShubhamRwt]

Yep I am working on that @ppatierno, trying to address all the comments and rebase. After that I will test things in a cluster to be sure everything works

[ppatierno]

@ShubhamRwt could you rebase against main, I merged another PR fixing some reconciliation issues on KafkaRebalance (You got conflicts as well).

[ppatierno]

@scholzj @PaulRMellor could you have another pass to check if Shubham addressed your comments as well?

[ShubhamRwt]

@scholzj Hi, can we run the pipeline tests in this?

[scholzj]

/azp run regression

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[ShubhamRwt]

The regression failure doesn't look related to this PR

[scholzj]

The regression failure doesn't look related to this PR

Ok, let's re-run it.

[scholzj]

/azp run regression

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[ShubhamRwt]

The test failing in the regression is passing for me locally even when I have rebased to the main branch. @see-quick Do you have any idea why it fails?

[ShubhamRwt]

@scholzj I have rebased the PR and made the minor fix. Do you have any other suggestions which I need to fix?