search

aquasecurity / trivy

Find vulnerabilities, misconfigurations, secrets, SBOM in containers, Kubernetes, code repositories, clouds and more
https://aquasecurity.github.io/trivy
Apache License 2.0
23.06k stars 2.28k forks source link

test(checks): Add automated bundle testing in pipeline #7031

Open simar7 opened 3 months ago

simar7 commented 3 months ago

Today when a PR is submitted in the checks repo, we have some basic testing that takes place as described in this workload. However, it offers no integration testing with Trivy itself.

For this to take place, we can do the following:

  1. Create a test environment with certain misconfigurations within the trivy-checks repo
  2. Create a bundle with the changes submitted in the PR within a GitHub Action pipeline
  3. Load this bundle into the latest release of Trivy, will be implemented via https://github.com/aquasecurity/trivy/issues/7029
  4. Make assertions on the checks themselves but also warrant that the bundle is loaded successfully.

By adding this automated layer of testing two problems will be tackled:

  1. Avoiding surprise breakages when upgrading trivy-checks usage within Trivy.
  2. Enabling automated scheduled releases of trivy-checks bundles without operator intervention. More details here: https://github.com/aquasecurity/trivy/issues/7030
nikpivkin commented 3 months ago

CI in trivy-checks is already doing integration testing of the checks bundle with the latest version of Trivy and seems to meet all the requirements described here. https://github.com/aquasecurity/trivy-checks/blob/main/scripts/verify-bundle.go

simar7 commented 2 months ago

CI in trivy-checks is already doing integration testing of the checks bundle with the latest version of Trivy and seems to meet all the requirements described here. https://github.com/aquasecurity/trivy-checks/blob/main/scripts/verify-bundle.go

Yes but there's currently no functional verification. We also lack testing against multiple versions of Trivy as we only test latest right now.