[Feature] Support managedBy for integration with MultiKueue

[mimowo]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

The support for the managedBy field which can delegate reconciliation from built-in controller, to a custom one.

The semantics of the field are:

whenever the value is set, and it does not point to the built-in operator, then skip reconciliation

• the field is immutable
• the field is not defaulted

For mpi-operator and training-operator we add validation to only support: null, reserved value (like kubeflow.org/mpi-operator), and kueue.x-k8s.io/multikueue.

Use case

Integration with MultiKueue (multi-cluster Kueue):

• simpler installation (just follow the standard installation path), otherwise only installation of RayJob CRDs is required
• support for mixed setup in one cluster - some RayJobs could be run by MultiKueue and some by the default operator

The integration was asked on the wg-batch Slack: https://kubernetes.slack.com/archives/C032ZE66A2X/p1731582793720509

Related issues

The managedBy field is already adopted by:

• k8s Job (beta in 1.32)
• JobSet (released in 0.6)
• MPI-operator (released in 0.6)
• training-operator (planned for release in Jan 2025)

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR! (not myself, but a team member is willing to work on that)

[mimowo]

cc @andrewsykim