defenseunicorns / uds-core

A FOSS secure runtime platform for mission-critical capabilities
https://uds.defenseunicorns.com
GNU Affero General Public License v3.0
50 stars 21 forks source link

Add conformance test suite #911

Open eddiezane opened 3 weeks ago

eddiezane commented 3 weeks ago

Is your feature request related to a problem? Please describe.

As we continue to deploy UDS to new environments, hardware, etc, it might be a good idea to add a conformance test suite that can validate compatibility beyond a surface level of "it deploys"

Further context, we deployed uds-core to an NVIDIA Jetson and it appears to successfully deploy, but the kernel image that NVIDIA ships is missing necessary kernel modules for Istio (known issue, https://uds.defenseunicorns.com/reference/uds-core/prerequisites/#istio).

Describe the solution you'd like

Describe alternatives you've considered

(optional) A clear and concise description of any alternative solutions or features you've considered.

Additional context

The goal here is to aid in cutting down the manual validation of UDS in a new environment/hardware.

We won't be able to predict ever scenario that needs to be tested for, but it would be good to have the bare necessities in place that we can then add new (edge) test cases as they creep up.

eddiezane commented 3 weeks ago

cc @sgettys

mjnagel commented 3 weeks ago

https://troubleshoot.sh/ could be interesting to look at here as well - they provide a specific walkthrough for preflights.

bdegeeter commented 3 weeks ago

Sonobuoy is another option to consider. https://sonobuoy.io/. It's used in a few distros e2e tests.