k8s-ci-robot
commented
3 weeks ago @AverageMarcus: This request has been marked as needing help from a contributor.
Guidelines
Please ensure that the issue body includes answers to the following questions:
- Why are we solving this issue?
- To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
- Does this issue have zero to low barrier of entry?
- How can the assignee reach out to you for help?
For more details on the requirements of such an issue, please see here and ensure that they are met.
If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.
In response to [this](https://github.com/kubernetes-sigs/image-builder/issues/1605):
>## Is your feature request related to a problem? Please describe.
>
>We've had a few instances now where changes introduced issues into existing targets, e.g.
>
>* https://github.com/kubernetes-sigs/image-builder/issues/881
>* https://github.com/kubernetes-sigs/image-builder/issues/879
>* https://github.com/kubernetes-sigs/image-builder/issues/828
>
>and most recently https://github.com/kubernetes-sigs/image-builder/pull/1596 caused an issue for all OS's that made use of init data that wasn't caught during manual testing due to focussing on Flatcar.
>
>## Describe the solution you'd like
>
>
>Where possible we should have automated tests in place to ensure that all currently supported providers and operating systems can build successfully with default values (this might be tricky in situations where there are required values).
>
>This issue is aimed at tracking the progress of this effort and should be updated as progress is made.
>
>Ideally, we should implement provider-specific tests that then build all OSs for that provider (e.g. using the `make build-raw-all` that should trigger all supported OSs).
>
>## Progress
>
>| Provider | Implemented | Comments |
>| --- | --- | --- |
>| ami | ❌ | I think the CAPA project had something that they (used to?) use for testing image builds. We should reach out to them and see if its something we can adopt. |
>| azure | Partial | We currently have two Prow jobs for Azure - `pull-azure-vhds` & `pull-azure-sigs`. These make use of [`ci-azure-e2e.sh`](https://github.com/kubernetes-sigs/image-builder/blob/main/images/capi/scripts/ci-azure-e2e.sh) but uses a pre-defined list of target ([`azure_targets.sh`](https://github.com/kubernetes-sigs/image-builder/blob/main/images/capi/azure_targets.sh)). Ideally we should try to update this to dynamically load all targets we support. |
>| digitalocean | ❌ | |
>| gce | Partial | We currently have the Prow job ` pull-image-builder-gcp-all` that uses [`ci-gce.sh`](https://github.com/kubernetes-sigs/image-builder/blob/main/images/capi/scripts/ci-gce.sh) but this is currently configured as an optional test and not automatically run on any PRs. We should update this to at least trigger when changes to GCE files are made, similar to the Azure ones. |
>| hcloud | ❌ | |
>| nutanix | ❌ | |
>| oci | ❌ | |
>| openstack | ❌ | |
>| outscale | ❌ | |
>| ova | ❌ | We recently had to remove this and need a new environment to run the tests in, see https://github.com/kubernetes-sigs/image-builder/issues/1508 |
>| powervs | ❌ | |
>| proxmox | ❌ | |
>| qemu | ❌ | Theoretically, this should be possible to test in our current infrastructure as its emulated. The problem might be the time it takes to run the tests though. |
>| raw | ❌ | Theoretically, this should be possible to test in our current infrastructure as its emulated. The problem might be the time it takes to run the tests though. |
>| vultr | ❌ | |
>
>## Describe alternatives you've considered
>
>
>Manual testing but this, as we have seen, isn't enough to catch all the issues.
>
>## Additional context
>
>
>Many of the providers will require actual cloud infrastructure to be able to run the build process. This is likely to be difficult to get unless the respective cloud providers are willing to donate resources for this purpose. (Please reach out to us if you are able to provide these resources)
>
>---
>
>/kind feature
>/lifecycle frozen
>/help
>/priority important-longterm
>/triage accepted
Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
Is your feature request related to a problem? Please describe.
We've had a few instances now where changes introduced issues into existing targets, e.g.
and most recently https://github.com/kubernetes-sigs/image-builder/pull/1596 caused an issue for all OS's that made use of init data that wasn't caught during manual testing due to focussing on Flatcar.
Describe the solution you'd like
Where possible we should have automated tests in place to ensure that all currently supported providers and operating systems can build successfully with default values (this might be tricky in situations where there are required values).
This issue is aimed at tracking the progress of this effort and should be updated as progress is made.
Ideally, we should implement provider-specific tests that then build all OSs for that provider (e.g. using the
make build-raw-allthat should trigger all supported OSs).Progress
pull-azure-vhds&pull-azure-sigs. These make use ofci-azure-e2e.shbut uses a pre-defined list of target (azure_targets.sh). Ideally we should try to update this to dynamically load all targets we support.pull-image-builder-gcp-allthat usesci-gce.shbut this is currently configured as an optional test and not automatically run on any PRs. We should update this to at least trigger when changes to GCE files are made, similar to the Azure ones. (see https://github.com/kubernetes-sigs/image-builder/issues/1601)Describe alternatives you've considered
Manual testing but this, as we have seen, isn't enough to catch all the issues.
Additional context
Many of the providers will require actual cloud infrastructure to be able to run the build process. This is likely to be difficult to get unless the respective cloud providers are willing to donate resources for this purpose. (Please reach out to us if you are able to provide these resources)
/kind feature /lifecycle frozen /help /priority important-longterm /triage accepted