Adding e2e tests for cluster provisioning when OSM is used

moadqassem commented 2 years ago

User Story In order to make sure that OSM is working as expected, we need to make sure that we test it thoroughly with every PR merge. That's why we need a proper infrastructure where we can build, run, integrate and test osm, to make sure everything is working as expected.

Acceptance Criteria

Add the proper scripts to handle the deployment of OSM
Make sure that OSM and OSM webhook are running and tested as well
Make sure that all OSM dependencies are in place
Run the e2e testing suite successfully with each master merge
Introduce e2e tests for all cloud provider that we support by OSM
Introduce e2e tests for all operating systems that we support by OSM
Introduce e2e tests for all kubernetes versions that we support by OSM

ahmedwaleedmalik commented 1 year ago

With https://github.com/kubermatic/machine-controller/pull/1369 in machine-controller, we switched the E2E tests to use OSM as default instead of the legacy user data. Hence, all the PRs in machine-controller are being tested against OSM as well.

Instead of reinventing the wheel, we can simply reuse those tests for OSM. The only significant change that we need for this is to build and use OSM image against the PR.

Prow job could like something like this:

  - name: pull-osm-e2e-aws
    decorate: true
    clone_uri: "ssh://git@github.com/kubermatic/operating-system-manager.git"
    extra_refs:
      - org: kubermatic
        repo: machine-controller
        base_ref: main
        clone_uri: 'ssh://git@github.com/kubermatic/machine-controller.git'
    labels:
      preset-aws: "true"
      preset-e2e-ssh: "true"
      preset-rhel: "true"
      preset-goproxy: "true"
      preset-kind-volume-mounts: "true"
      preset-docker-mirror: "true"
      preset-kubeconfig-ci: "true"
    spec:
      containers:
        - image: quay.io/kubermatic/build:go-1.20-node-18-kind-0.18-7
          command:
            # Build OSM container image and then call `./hack/ci/run-e2e-tests.sh` from machine-controller.
            - "./hack/ci/run-e2e-tests.sh"
          args:
            - "TestAWSProvisioningE2E"
          env:
            - name: CLOUD_PROVIDER
              value: aws
          securityContext:
            privileged: true
          resources:
            requests:
              memory: 7Gi
              cpu: 2
            limits:
              memory: 7Gi

NOTE: We should ideally take https://github.com/kubermatic/machine-controller/issues/1626 in account as well while working on this ticket.

ahmedwaleedmalik commented 1 year ago

@moadqassem WDYT? I think this should significantly reduce the effort required to integrate and maintain E2E tests for OSM. We don't really need to implement anything per OS, cloud provider basis, etc. since MC already is taking care of all of those combinations.

moadqassem commented 1 year ago

@moadqassem WDYT? I think this should significantly reduce the effort required to integrate and maintain E2E tests for OSM. We don't really need to implement anything per OS, cloud provider basis, etc. since MC already is taking care of all of those combinations.

Yes that sounds good for me as well. I will close the epic.

kubermatic-bot commented 2 months ago

Issues go stale after 90d of inactivity. After a furter 30 days, they will turn rotten. Mark the issue as fresh with /remove-lifecycle stale.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kubermatic / operating-system-manager

Adding e2e tests for cluster provisioning when OSM is used #105