New way of installation

[a-thaler]

Description

Motivation: Having a fast parallel & out of the cluster Kyma deployment.

Problems of current installation:

• Installation duration takes long (a lot can happen in between, development cycle, user experience)
• Unreliable (running jobs inside charts, helm hooks are not well interruptable. Interrupts will happen by design by apiserver losts)
• Feature gaps in overrides, often not possible to use available helm values features
• Development unfriendly (apply changes from local will require a custom installer image, hard to debug)
• security (serviceaccount has toooo many permissions)

Goal:

• have new install/deploy commands being proven stable
• support K3s (including windows)
• works with any kyma revision without required local checkout
• upgrade does not require repetitive config settings (config once and keep it over later upgrades)
• CI pipelines are proving stability
• No compatibility (we decided to have no compatibility phase in the new installer
• Based on plain values.yaml
• Improved development speed by running on client-side and allowing single component deploys
• Running outside the cluster
  • Applying resources fast, avoid jobs and move them out of charts
  • Re-try loop controlled by client
  • Idempotent
• CRDs as pre-step

Open Features

• Networking

  • [x] Sample evaluation Domain & TLS cert for remote cluster
  • [x] CoreDNS rewrite patch (https://github.com/kyma-incubator/hydroform/issues/234)
  • [x] CoreDNS registry patch (https://github.com/kyma-project/hydroform/pull/280)
  • [x] Local cluster dev domain blocked (https://github.com/kyma-project/kyma/issues/10881)

• SKR

  • [x] Update kubeconfig passing options on the API (https://github.com/kyma-project/hydroform/issues/256)
  • [x] Overall installation status (https://github.com/kyma-incubator/hydroform/issues/231)
  • [x] implement kyma source downloader and expose it in the lib api (Lib: https://github.com/kyma-incubator/hydroform/issues/225 / CLI: https://github.com/kyma-project/cli/issues/809)

• Installation improvements

  • [x] uninstalling CRDs (https://github.com/kyma-incubator/hydroform/issues/214)
  • [x] Use reuse value feature for upgrades (https://github.com/kyma-incubator/hydroform/issues/188)
  • [x] uninstall has to remove Kyma specific namespaces (https://github.com/kyma-incubator/hydroform/issues/181)
  • [x] use Helm library in Hydroform to store installation state (don't access Helm specific secrets via Kubernetes client) (https://github.com/kyma-incubator/hydroform/pull/232)
  • [x] Resilience: Check component pending state when upgrading (https://github.com/kyma-incubator/hydroform/issues/196)
  • [x] Explore all jobs in charts and challenge implementation (with follow-ups)
  • [x] Avoid jobs in charts: rework cert management (https://github.com/kyma-project/kyma/issues/10336)
  • [x] Avoid jobs in charts: cluster-essentials reworking (https://github.com/kyma-incubator/hydroform/issues/177)
  • [x] No server side-state: Remove components-list and installation status from config-map in cluster and use HELM release secret (by adding labels) to retrieve an overview of installed Kyma components. (https://github.com/kyma-incubator/hydroform/pull/230)

• API improvements

  • [x] Refactor overrides and interceptors (https://github.com/kyma-incubator/hydroform/issues/201)
  • [x] Components as data structure instead of file (solved together with: No server side-state https://github.com/kyma-incubator/hydroform/pull/230)

• CLI

  • [x] User overrides not intercepted (https://github.com/kyma-project/cli/issues/904)
  • [x] Adapt to the callback progress update model (https://github.com/kyma-project/cli/issues/859)
  • [x] Add support for all k3d parameters in CLI (https://github.com/kyma-project/cli/issues/787)
  • [x] quality: HELM --set flags should be supported by Kyma CLI (https://github.com/kyma-project/cli/issues/725)
  • [x] K3d registry support added to provisioning command (https://github.com/kyma-project/cli/issues/825)
  • [x] CLI integration test on K3s (https://github.com/kyma-project/cli/issues/671)
  • [x] Change default location / name of workspace directory to avoid collisions with the user file-structure (https://github.com/kyma-project/cli/issues/854)
  • [x] single components:
  • support single component list (CLI supports flags for defining components) (https://github.com/kyma-project/cli/issues/826)
  • Add component sub-command to CLI: list components, show details (metadata) of a component, allow adding / removing a Kyma component (https://github.com/kyma-project/cli/issues/827)
  • [x] istio chart not working with k3s, fix it (https://github.com/kyma-incubator/hydroform/issues/171)

• Other

  • [x] Cleanup: Remove anything for compatibility (old override support, static admin/pwd, ...)
  • [x] Stablize Gardener CI pipeline (https://status.build.kyma-project.io/?job=pre-master-kyma-gardener-azure-kyma-alpha)

---

Tasks:

• hydroform
  • [x] logger refactoring (https://github.com/kyma-incubator/hydroform/issues/169) Don't pass logger-function between involved object, use logger API
  • [x] exception handling (https://github.com/kyma-incubator/hydroform/issues/170)
  • [x] API field to toggle atomic upgrades (https://github.com/kyma-incubator/hydroform/issues/195)
  • [x] defaultNamespace field should become optional in components.yaml (use kyma-system as default if undefined - see https://github.com/kyma-incubator/hydroform/pull/215).
  • [x] deployments have to behave idempotent and an interrupted deployment should not become a blocker for the next deployment (https://github.com/kyma-incubator/hydroform/issues/174)
  • [ ] Update kyma alpha uninstallation pipeline to latest kyma cli changes (https://github.com/kyma-project/test-infra/issues/3125)
• CLI
  • [x] sources support (https://github.com/kyma-project/cli/issues/685) Sources will specify primarily the source of the components list (can be a release version, local sources etc.).
  • [x] K3S Windows Support (https://github.com/kyma-project/cli/issues/695)
  • [x] Support custom domain-name and certficiates (also https://github.com/kyma-project/cli/issues/681)
  • [x] values config (https://github.com/kyma-project/cli/issues/690) Customer should be able to provide: a) values file -v $pathToValuesFile b) generic set param -s another.override.abc=123 -s another.override.xyz=987 (has precedence over values defined in values file)
  • [x] component list defined (containing prereqs, default installation to defined kyma ns)
  • [x] overrides for k3s (https://github.com/kyma-project/kyma/issues/10359)
  • [x] K3S Mac / Linux support (https://github.com/kyma-project/cli/issues/670)
  • [x] upgrade support / version check (https://github.com/kyma-project/cli/issues/691)
  • [x] uninstall command working should be consistent with deploy (https://github.com/kyma-project/cli/issues/692)
  • [x] metadata command (https://github.com/kyma-project/cli/issues/698)
  • [x] show a summary at the end of the Kyma deployment (https://github.com/kyma-project/cli/issues/702)
  • [x] support override/values files and component-list file from remote locations (https://github.com/kyma-project/cli/issues/701)
  • [x] adjust parameter names for overrides (https://github.com/kyma-project/cli/issues/704)
  • [x] custom certificate has to be provided as file (https://github.com/kyma-project/cli/issues/706)
  • [x] align logging in CLI with Kyma logging recommendation (https://github.com/kyma-project/cli/issues/708)
  • [x] support error messages if a failure occurred during deploy / undeploy procedure (see https://github.com/kyma-project/cli/issues/730)
  • [x] Use K3s specific component list (https://github.com/kyma-project/cli/issues/740)
  • [x] Move value resolution to hydroform (https://github.com/kyma-project/cli/issues/756)
  • [x] Flag to disable atomic upgrades (https://github.com/kyma-project/cli/issues/755).
  • [x] Set workspace path to $GOPATH/src/kyma-project/kyma if this folder exists and --source local is set (https://github.com/kyma-project/cli/issues/776)
  • [x] Change flag --no-deploy (is deprecated) to recommended parameter name (--disabled) ink3d cluster create command (https://github.com/kyma-project/cli/pull/774)
• general
  • [x] improve uninstallation by forcing ns deletion (https://github.com/kyma-incubator/hydroform/issues/173)
  • [x] fix issue with Kubernetes DNS resolver when using local.kyma.dev as domain (https://github.com/kyma-project/cli/issues/697)
  • [x] support server-args for K3s (https://github.com/kyma-project/cli/issues/700)
  • [x] mask sensitive data in verbose console output (https://github.com/kyma-project/cli/issues/703)

The rollout of the new way of installation will be covered in https://github.com/kyma-project/kyma/issues/10812

[a-thaler]

the new way of installation is working, next phase is to role it out. For that, see https://github.com/kyma-project/kyma/issues/10812