Open viccuad opened 1 year ago
As talked on daily, piling up here several features wanted.
The majority of the tests are tightly coupled with the helm chart code, hence the test code could be part of the chart repo. The only tests not tightly coupled would be for chart dependencies (such as open-telemetry, grafana, etc), and we only need minimum integration tests for those. For vendored chart dependencies such as policy-reporter-ui are tightly coupled and we can include integration tests in the chart repo.
My list of desired features:
helm lint
or chart-testing lint
.<chart>/templates/tests/
entries. See https://helm.sh/docs/topics/chart_tests/. This allows for jobs to run on a cluster with the chart installed. This ships with the chart, can be run by users to certify the chart is working as intended.Example of a Helm chart with unit, integration tests: https://github.com/hashicorp/vault-helm, albeit DIY.
helm
hashelm fetch
, that obtains a chart from a URL or repo, and saves it as tar.gz. For all operations likehelm install
,helm upgrade
, helm accepts a local folder or a tar.gz as a chart.I would like to separate the setup of the end-to-end tests and the run of tests, where:
helm fetch
all needed charts, from whatever we have defined. For example, we can support as input a yaml file, or env vars, etc, where we list charts upstream names, their location (a repo, a url for a tar.gz, a folder, etc), and our test name for them:(kubewarden-controller-1.7.0, https://charts.kubewarden.io, kubewarden-controller)
: fetch thekubewarden-controller-1.7.0
chart from thaturl
and save it askubewarden-controller
.helm fetch --untar
, orhelm fetch --untar --devel
if it has-rc
on the name. This script doesn't need much checks, just iterate through everything and do the helm fetch.kubewarden-controller
. We can use that chart name in all our tests.kubewarden-controller-1.7.0-rc1
and runs some tests, it performshelm install kubewarden-controller && run test
.kubewarden-controller-1.7.0-rc1
is upgradable to a yet to be released version, we would perform thenhelm upgrade kubewarden-controller kubewarden-controller-next
.helm install kubewarden-controller-ga && helm upgrade kubewarden-controller
.I think this brings some nice behaviour:
charts-under-test
folder manually. This is of great help for local development: I can easily consume a helm chart from a git branch, and consume my local helm chart folder by making a symlink into thecharts-under-test
folder.Improvements
A somewhat similar approach could be used for configuring the charts. For example, a specific combination of chart versions may need to consume a container image
:latest
and set a specific config. It would be easier to drop adefault-values-chart-under-test/values.yaml
with:This can help when using the end-to-end tests when developing, as one can just drop the values.yaml, and the helm-chart. The same approach can be used recursively inside of the tests, with each test being a folder, where there's resources to kubectl apply, values.yaml to upgrade to, etc.