search

cncf / crossplane-conformance

Crossplane Conformance Program
https://cncf.io/projects
Apache License 2.0
16 stars 13 forks source link

Automate Conformance Tests #25

Open luebken opened 2 years ago

luebken commented 2 years ago

While investigating the Crossplane performance tests, it became apparent that it currently requires a lot of manual effort to run these tests. To make these tests run more regularly and on new releases of providers it would be great to automate them as much as possible.

negz commented 2 years ago

I think I know what you mean, but it would be useful to be explicit here. Currently:

The devil is in the details though; the part that is not automated about provider conformance tests is requiring that all managed resources be manually created so that the tests can subsequently inspect whether they can become "ready". That one task is difficult to automate, but is also very labor intensive for providers with more than a handful of managed resources.