search

kyma-project / kyma

Kyma is an opinionated set of Kubernetes-based modular building blocks, including all necessary capabilities to develop and run enterprise-grade cloud-native applications.
https://kyma-project.io
Apache License 2.0
1.52k stars 405 forks source link

Community modules in managed and non-managed environment #18450

Open pbochynski opened 11 months ago

pbochynski commented 11 months ago

Description

Provide a way to install community modules using the Kyma Dashboard that is consistent with adding fully managed modules. UI should be simple and guide the user to use the best available option:

Other options should be also available, but in advanced mode (installing manageable module without KLM - opt-out from full support)

Reasons

Kyma comes with a set of Kubernetes building blocks (called modules). Modules are independent: can be added and removed in the runtime, and have an independent release cycle. Each module provides and a manager (Kubernetes operator) and a default configuration (custom resource). Some modules are available in the managed SAP Kyma Runtime with full support (SLA, automatic upgrades). Other modules (community modules) are offered only with community support (no SLA, manual upgrades, possible breaking changes). In the managed clusters the Kyma Lifecycle Manager (KLM) is responsible for installation and automatic upgrades of supported modules. KLM is running centrally in the Kyma Control Plane and is configured with Kyma custom resource in the namespace kyma-system located in the managed cluster. Users who want to use some managed modules have to add their names to the Kyma CR spec. The community modules cannot be managed by the KLM and should be installed with the desired version using kubectl with the provided Kubernetes manifests (manager deployment and default configuration). The situation gets more complicated when users want to install the "manageable" modules in their own (not managed) cluster (like k3s) or if they want to opt-out from full management in the managed cluster (install module manager with kubectl without using KLM).

Having a list of available modules with clear indicators if the module is/can be fully managed or not, and a consistent convenient installation process is a must-have.

Acceptance Criteria

Module scenarios

Installation

Manageable modules should be installed with KLM if Kyma CR is available (managed cluster). The direct installation (without KLM) should be possible for manageable modules but should be harder to find and guarded by warnings. It should not be possible to install the module directly if it is already a managed module. In the open source scenario (not managed cluster) all modules should be installed directly with the provided Kubernetes manifests.

Note: direct installation and deletion are idempotent actions - and can be repeated several times without side effects. It should be possible to trigger installation or deletion several times (finish an interrupted/incomplete action by applying/deleting again, or upgrade the module manager by applying a new version)

Upgrade

For managed modules, it happens automatically. For not managed modules user has to apply the module manifest again. Default module configuration (default CR) should not be applied for updates. The module operator in the new version should handle previously created configurations. For both cases (managed and manual) it is essential to check what is the current version of the module manager (operator) installed in the cluster. UI should compare the desired version with the version that is deployed in the cluster. If the version does not match it is the indicator that should be visible to the user. It means that for managed components KLM hasn't finished the upgrade yet, for not managed modules it should encourage the user to perform the upgrade (press the apply button).

Deletion

Module manager should not be undeployed before any managed resource (including module configuration) still exists. Otherwise, there is a good chance that the orphan resources will stay in the cluster forever. Therefore each module should declare the list of managed resources (group, version, kind) and the user should actively delete those resources before the module is finally deleted (manager undeployed). This step should be executed regardless if the module is managed by KLM or installed manually.

Warning: when you delete the managed module configuration before the module is removed from Kyma CR spec, the KLM can try to recreate it and the module installation starts again. In that case, module configuration should be removed by KLM or the customResourcePolicy should be changed to Ignore (from the default CreateAndDelete)

flowchart TD
    N([Start: Choose Module to Delete]) --> MO{"Do managed 
    resources exist?"}
    MO -->|No| O{"Is the Module 
    Managed?"}
    MO -->|Yes| DMO{Continue?}
    DMO -->|Yes| DM[Delete managed resources]
    DMO -->|No| E
    DM --> MO
    O -->|Yes| P["Delete via KLM
                  Remove module 
                  from Kyma CR"]
    O -->|No| T["Direct Deletion 
    of module manager 
    Kubernetes resources"]
    P --> E([End])
    T --> E

Managed modules should be deleted by KLM and the deletion process should be triggered by removing module entry from Kyma CR spec. Not-managed modules should be removed directly by deleting Kubernetes resources included in the module manager manifest.

Status handling

With automatic and manual lifecycle management it is not so easy to determine if the module is installed and what is the status. Here are some rules that can be applied to determine module status and possible actions:

Diagram

flowchart TB
    A[Start: Choose Module to Install] --> B{Is the Cluster Managed?}
    B -->|Yes| C[Use Kyma Lifecycle Manager]
    B -->|No| D{Is the Module Manageable?}
    C --> E[Modify Kyma CR for Installation]
    E --> F[Automatic Updates via KLM]
    D -->|Yes| G[Install Manageable Module Directly]
    D -->|No| H[Install Community Module with kubectl]
    G --> I[Option: With KLM for Support]
    G --> J[Option: Without KLM - Opt-out from Full Support]
    H --> K[Manual Upgrades, Possible Breaking Changes]
    I --> L[Managed with SLA, Automatic Upgrades]
    J --> M[Warning: Limited Support]
flowchart TB
    U[Start: Check Module Status] --> V{Is Module in Kyma CR?}
    V -->|Yes| W[Managed and Enabled]
    V -->|No| X{Does Default CR Exist?}
    X -->|Yes| Y[Check if CR has Deletion Timestamp]
    Y -->|No Timestamp| Z[Module is Enabled]
    Y -->|Has Timestamp| AA[Module is Deleting]
    X -->|No| AB[Module is Removed]
    W --> AC[Ready if CR Applied and Manager at Desired Version]
    Z --> AC
    AA --> AC

Attachments

github-actions[bot] commented 8 months ago

This issue has been automatically marked as stale due to the lack of recent activity. It will soon be closed if no further activity occurs. Thank you for your contributions.

valentinvieriu commented 7 months ago

From UI Perspective - we decided to refactor our UI to match the PRoposed Designs using the Existing implementation with KYMA CR (https://github.com/kyma-project/busola/issues/2624 )

Once the new modules catalog will be ready, we will switch to the new API. We are waiting for the feedback when the new JSON based API will be ready and we can switch from reading Module Templates to display Module Metadata.

tobiscr commented 4 months ago

@tehret77 - JFYI, this feature will make the management of Kyma community modules more convenient in the future.

varbanv commented 3 months ago

Due date shifted to reflect additional work that is needed for UI/CLI, etc.

janmedrek commented 3 months ago

The proposal for all the module catalogue changes is almost finalized here: https://github.com/kyma-project/lifecycle-manager/issues/1681

Right now, we are waiting for input from the Busola and CLI teams to shape the decision and proceed with the implementation tweaks.

After that, it is all about adapting the UI and pipelines to match the picture and we are good-to-go with releasing the Community Modules as a whole.

janmedrek commented 3 months ago

We would also need some documentation for the community-contributed modules, such as what metadata to provide and where it should be placed.

Also, a modulectl command for creating a module template would be welcome.

janmedrek commented 3 months ago

Extended with the documentation issue: https://github.com/kyma-project/lifecycle-manager/issues/1812

janmedrek commented 1 month ago

Update: