Add Helm E2E tests - Githubissues

ArangoGutierrez commented 2 weeks ago

What would you like to be added: Now that we are running the E2E with Kind and we have more flexibility over our E2E infra we should include a small test suite for running via Helm deployment.

Why is this needed: Currently, E2E deploys objects from code, leaving Helm charts untested. E2E could pass but Helm charts be broken and we won't tell until a manual test.

/help /good-first-issue

k8s-ci-robot commented 2 weeks ago

@ArangoGutierrez: This request has been marked as suitable for new contributors.

Guidelines

Please ensure that the issue body includes answers to the following questions:

Why are we solving this issue?
To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
Does this issue have zero to low barrier of entry?
How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-good-first-issue command.

In response to [this](https://github.com/kubernetes-sigs/node-feature-discovery/issues/1769): >**What would you like to be added**: >Now that we are running the E2E with Kind and we have more flexibility over our E2E infra we should include a small test suite for running via Helm deployment. > >**Why is this needed**: >Currently, E2E deploys objects from code, leaving Helm charts untested. E2E could pass but Helm charts be broken and we won't tell until a manual test. > >/help >/good-first-issue Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

THUzxj commented 2 weeks ago

Hi, I would like to work on this. I might need to spend some time understanding this project first. Thanks!

TessaIO commented 2 weeks ago

Hi, I would like to work on this. I might need to spend some time understanding this project first. Thanks!

lmk if you need any guidance with that!

THUzxj commented 1 week ago

Hi @TessaIO , thank you for offering help. I am struggling to run the e2e test locally. Since the Makefile doesn't provide scripts to create the testing environment, I'd like to use Kind. Here is my practice to run the e2e test:

kind create cluster
# in source dir
make image-all
kind load docker-image  registry.k8s.io/nfd/node-feature-discovery:xxxx
make test-e2e

However, I am faced with the problem that Kind still complains "Failed to pull image" error and I am investigating this.

Thus I would appreciate it if you could answer these questions: Do you run the e2e tests locally when developing? If so, do you have some recommended ways?

TessaIO commented 1 week ago

Hi @THUzxj, Your setup is correct. Few params need to be tweaked here. When u run make test-e2e you can see this code snippet in the Makefile file:

e2e-test:
    @if [ -z ${KUBECONFIG} ]; then echo "[ERR] KUBECONFIG missing, must be defined"; exit 1; fi
    $(GO_CMD) test -timeout=1h -v ./test/e2e/ -args \
        -nfd.repo=$(IMAGE_REPO) -nfd.tag=$(IMAGE_TAG_NAME) \
        -kubeconfig=$(KUBECONFIG) \
        -nfd.e2e-config=$(E2E_TEST_CONFIG) \
        -nfd.pull-if-not-present=$(E2E_PULL_IF_NOT_PRESENT) \
        -ginkgo.focus="\[k8s-sigs\/node-feature-discovery\]" \
        -ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER) \
        -ginkgo.v \
        $(if $(OPENSHIFT),-nfd.openshift,)
    if [ "$(E2E_TEST_FULL_IMAGE)" = "true" ]; then \
        $(GO_CMD) test -timeout=1h -v ./test/e2e/ -args \
            -nfd.repo=$(IMAGE_REPO) -nfd.tag=$(IMAGE_TAG_NAME)-full \
            -kubeconfig=$(KUBECONFIG) \
            -nfd.e2e-config=$(E2E_TEST_CONFIG) \
            -nfd.pull-if-not-present=$(E2E_PULL_IF_NOT_PRESENT) \
            -ginkgo.focus="\[k8s-sigs\/node-feature-discovery\]" \
            -ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER) \
            -ginkgo.v \
            $(if $(OPENSHIFT),-nfd.openshift,); \
    fi

As you can see that nfd.tag is equal to the IMAGE_TAG_NAME which is equal to the value of this command:

$(shell git describe --tags --dirty --always)

So in a nutshell, you're e2e-test command would try to pull node-feature-discovery:smth-random-here You have two ways here: 1- If you want to run tests against the latest version you need to change version in you Makefile to point 0.16.1 or whatever u want 2- Build and push your own image using:

make image
make push

and then run your e2e-tests

/assign @THUzxj

marquiz commented 1 week ago

However, I am faced with the problem that Kind still complains "Failed to pull image" error and I am investigating this.

You need to use the E2E_PULL_IF_NOT_PRESENT=true so that your test cluster doesn't try to pull that (local) image:

E2E_PULL_IF_NOT_PRESENT=true make e2e-test

kubernetes-sigs / node-feature-discovery

Add Helm E2E tests #1769

Guidelines