[ModuleCatalog] Implementation of a Dedicated Controller for Syncing `ModuleTemplate` and `ModuleReleaseMeta` CRs

[jeremyharisch]

Description:

The synchronization process of ModuleTemplate CRs to all SKR clusters is currently embedded within the Kyma reconcile loop. We have decided to expand this functionality to include ModuleReleaseMeta CRs and to relocate these synchronization tasks to a new, dedicated controller. This approach is intended to enhance performance by offloading the sync operations from the main Kyma reconcile loop.

Detailed decision ticket: https://github.com/kyma-project/lifecycle-manager/issues/1815

Current Implementation:

The existing synchronization of ModuleTemplate CRs is integrated into the Kyma reconcile loop, detailed here.

Objective:

• Develop a Dedicated Sync Controller:

  • This new controller will handle the synchronization of both ModuleTemplate and ModuleReleaseMeta CRs from the KCP cluster to all SKR clusters.
  • It will manage the lifecycle of these CRs—handling creation, updates, and deletions—and will ensure that all corresponding Kyma CRs are requeued accordingly.

• Performance Optimization:

  • By isolating the synchronization process, we aim to reduce the overhead in the Kyma reconcile loop, which currently dedicates approximately 20% of its time to sync operations.

• Testing and Validation:

  • Comprehensive E2E and integration testing will be conducted to ensure the new controller operates effectively and integrates smoothly with existing systems.
  • Adjustments will be made to existing tests that are affected by this change.

Benefits of Separation of Concerns:

• Enhanced Performance: Removing the sync tasks from the Kyma controller allows for more focused and efficient reconciliation processes.
• Reduced Complexity in Core Operations: Segregating the synchronization functionality simplifies the main Kyma reconcile loop, making it easier to maintain and scale.

Potential Challenges:

• System Complexity: Introducing a new controller adds a layer of complexity to the overall architecture.
• Race Conditions: Separate controllers working on related resources could lead to conflicts, although these are generally manageable within Kubernetes through established patterns.

Acceptance Criteria:

• [ ] Implementation of the New Controller: The controller successfully synchronizes ModuleTemplate and ModuleReleaseMeta CRs across SKR clusters.
  • [ ] Make sure code is easy testable, stick to our current implementation patterns.
  • [ ] If a ModuleTemplate or ModuleReleaseMeta CR gets created, updated or deleted, all Kyma CRs should be requeued, since all are affected automatically.
  • [ ] Name of controller: SyncResources
  • [ ] Cover controller with basic metrics
  • [ ] Update only affects the .status.condition
• [ ] Performance Improvements Documented: Metrics demonstrating the reduced load on the Kyma reconcile loop.
• [ ] Comprehensive Test Coverage: The new functionality is fully covered by E2E and integration tests.
• [ ] Documentation: Updated documentation reflecting the new controller’s role and functionality within the ecosystem.

[Tomasz-Smelcerz-SAP]

I think we need to refine on the AC: "Name of controller: SyncResources". The SyncResources is too generic and vague. I suggest to use just: catalogsync