[registry-operator]: operator/controller framework

[shanduur]

Context and Problem Statement

The registry-operator should be a critical component in managing CNCF Distribution Registry instances efficiently. However, selecting the appropriate base operator/controller framework is pivotal for ensuring the scalability, maintainability, and extensibility of the registry-operator.

Considerations:

• the framework must provide robust functionality to streamline deployment, scaling, and management of container image registries, aligning with the objectives of the registry-operator;
• it should offer ease of development and maintenance, allowing the maintainers to efficiently manage and extend the operator's capabilities as per evolving requirements;
• community support, documentation quality, and integration capabilities with other Kubernetes ecosystem components are crucial factors to weigh in our decision-making process.

Considered Options

• Kubebuilder
  • Description: A framework for building Kubernetes operators in Golang, built on top of controller-runtime.
  • Pros: Mature, large community, easy to get started, good documentation.
  • Cons: N/A
• Operator-SDK
  • Description: A CLI tool for building Kubernetes operators, built on top of controller-runtime and Kubebuilder.
  • Pros: Integrates with OLM.
  • Cons: Additional complexity and abstraction around Kubebuilder.
• Kube.rs
  • Description: A framework for building Kubernetes operators in Rust.
  • Pros: Memory safety, potentially better performance.
  • Cons: Less mature than Kubebuilder and Operator-SDK, smaller community and ecosystem, team not experienced with Rust.

[shanduur]

Both Operator-SDK and Kubebuilder are state of the art tools. However my personal preference lays with Kubebuilder:

• it's easier to get started with it;
• it's the backbone of Operator-SDK, and thus the community of the later is also contributing to and working on the former;
• it is possible to use Operator-SDK with projects started with Kubebuilder.

Discard my comment, the project generated by two tools is almost identical, so no need to worry about OSDK.

[purpleturtll]

I generated scaffolding with both kubebuilder and operator-sdk to check what is the difference.

List of what operator-sdk generates that is not generated by kubebuilder:

• OLM bundle specific Makefile tasks and variables
• Initial Scorecard configuration

Basic development flow is the same for both projects:

• kubebuilder create api --group ship --version v1beta1 --kind Frigate or operator-sdk create api --group ship --version v1beta1 --kind Frigate
• make docker-build to build image
• make docker-push to push image
• make deploy to deploy on cluster

If we want to support OLM I don't see a reason to not scaffold with operator-sdk. If we get automation in place to test the OLM installation and run Scorecard tests I don't think there is any added complexity (apart from thinking about OLM installation if something breaks).

[purpleturtll]

Do we want to seriously consider Rust? I think this would change the scope of the project for initial contributors from writing an operator to learning Rust and writing an operator? I would be up for the challenge but I think we would need to be unanimous in this decision.

[shanduur]

As I really would like to use Rust, I am nowhere near to be proficient in it, and writing the operator would probably require using Tokio, which is one extra level of complexity.

Also, if no one of us is proficient, then I feel like the quality of code and development speed would suffer.

I also tested the OSDK vs KB, and yeah, seems fine to me to use the former one 🔥

CC: @majabojarska @malgorzatadutka

[malgorzatadutka]

I'd like to be proficient in Go rather than starting with new language. The second thought Kubebuilder and Operator-SDK are more mature.

[shanduur]

Looks like we have a consensus - let's use Operator-SDK to bootstrap the repository!