Provide tooling for automated Kyma lifecycle and subscription management

varbanv commented 9 months ago

Description

Users should be able to fully automate Kyma related tasks in their CI/CD pipelines. This starts with Kyma instance provisioning, includes deployment of necessary artifacts and test executions, and ends with Kyma instance removal and associated service instance cleanup. All of this should not require much additional effort from the end user and should be as easy as possible to configure.

Context

Problem

Right now, users can automate the Kyma instance creation using the new Terraform provider, the cloud orchestrator tools, or the btpcli directly, however, the next step of accessing the cluster requires user interaction.

And finally, there are a number of scenarios that could prevent a cluster from being deleted and would require user interaction to complete. While the second and third problems can be solved with some coding and additional resources, we want to provide ease-of-use in order to improve the perception and adoption of Kyma.

Benefits

For customers:

speed up development and release cycles
implement best practices around CI/CD
reduce configuration drift
reduce dependency on long-living Kyma instances
reduce cost by only using Kyma instances when needed Example usecase scenario that should be possible to run in automated way

For us:

increase adoption
enable customers to treat Kyma instances as disposable assets and reduce risk
enable users to reduce cost

Proposed solution

Design and implement a new set of kyma CLI commands that helps developers who use managed kyma runtimes within BTP ecosystem with development of their CAP applications. CLI commands should be designed with automation flow in mind (No user context should be required),

The commands should cover:

automated lifecycle management of kyma runtime lifecycle (provisioning, deprovisioning)
getting access to freshly created cluster
attaching BTP service instances to kyma runtimes via service manager
eliminating manual actions (i.e mapping hana instance, establishing trust for custom IAS tenant, etc..)

Acceptance criteria

[ ] Users can create a Kyma instance and access it without any user interaction.
[ ] Users can completely remove an existing Kyma instance without any user interaction, even if there are modules "stuck" in deletion.
- [ ] https://github.com/kyma-project/infrastructure-manager/issues/103
[ ] Provided solutions satisfy all security and compliance SAP standards (for example: no non-expiring access).
[ ] Documentation and guides that cover the full end-to-end CI/CD scenario

Attachments

Idea for assisted kyma provisioning

Disper commented 8 months ago

What would be the exact/examples of scenarios around provisioning?

kwiatekus commented 7 months ago

Similar request to log into kyma in headless mode https://github.tools.sap/kyma/backlog/issues/2518

kwiatekus commented 7 months ago

https://github.tools.sap/kyma/backlog/issues/2660#issuecomment-2104608

pbochynski commented 7 months ago

One possible solution is: https://github.com/kyma-project/kyma/issues/18305

kwiatekus commented 7 months ago

What would be the exact/examples of scenarios around provisioning?

@Disper Additional config decribing a "system issuer" should be collected from user and sent to KEB. Further on, provisioner uses this data to deploy the OICD config object and enable the shoot-oidc-service extension. https://github.tools.sap/kyma/backlog/issues/2660#issuecomment-3658944

Provisioner should add cluster role binding to principal represented by tokens issued by the system issuer

kwiatekus commented 6 months ago

Provisioning kyma runtime via BTP CLI https://blogs.sap.com/2022/02/24/creating-sap-btp-kyma-runtime-via-the-sap-btp-cli/

Cloud orchestrator: https://pages.github.tools.sap/cloud-orchestration/

kwiatekus commented 6 months ago

Another approach would be to implement OSB's create_binding API on the KEB side, so that after kyma instance is created one could immediately create binding and use the binding data to access kyma runtime. This has strong benefits:

complies with OSB specification and with UX around other BTP services (i.e create instance of hana, create binding to access it)
doesn't require additional OIDC config for simple use cases for short-living disposable kyma runtime (exactly for CI/CD flows)

kwiatekus commented 2 months ago

For users to be able to deploy their code w/o additional subscription (in the DEV mode) we aim to separate deocker-regisrty into separate DEV module to be used also outside of serverless usage https://github.com/kyma-project/kyma/issues/18555#issuecomment-2068633912

kwiatekus commented 2 months ago

As of today, the new prototype commands (developed in https://github.com/kyma-project/cli/tree/v3) allow to

provision managed kyma runtime within SAP BTP subaccount
create hana db instance in an automated way (incl. binding and mapping to kyma).
create references to shared instances

Getting access to new kyma runtime vi kyma CLI (in 100% automated scenario) will be possible after implementig https://github.com/kyma-project/kyma/issues/18305

kwiatekus commented 2 months ago

Customizable OIDC configurations (required for this epic) are being discussed within https://github.com/kyma-project/infrastructure-manager/issues/134

kwiatekus commented 2 months ago

We started working on cli part:

kyma cli command that generates kubeconfig for CI/CD workflow https://github.com/kyma-project/cli/issues/2093
enhancing cli provisioning command to onclude extra oidc config https://github.com/kyma-project/cli/issues/2115 (dependency to KEB https://github.com/kyma-project/kyma-environment-broker/issues/423)
kyma cli command that helps creating a serviceaccount based kubeconfig https://github.com/kyma-project/cli/issues/2036

ETA - on track

kwiatekus commented 2 weeks ago

Currently @tobiscr and the team is planning a migration from provisioner to KIM. To start initial testing of the tooling (provisioning and getting access in automated scenario) it would be necessary to switch provisioning of new clusters via kyma infrastructure manager.

tobiscr commented 2 weeks ago

Plan is to have a working KIM version on DEV latest on end of July 2024

kwiatekus commented 1 week ago

We have synced with @tobiscr : The latest estimate for KIM delivery on prod is end of August 2024.

We could start testing the e2e provisioning (with additional oidc) via kyma cli on DEV only after necessary integration of KEB to KIM is implemented on DEV. @kyma-project/gopher PTAL https://github.com/kyma-project/kyma-environment-broker/issues/423#issuecomment-2194680153

kyma-project / kyma