search

operator-framework / operator-courier

Build, verify and push operators
Apache License 2.0
41 stars 53 forks source link

test: OMPS integration tests #161

Closed estroz closed 4 years ago

estroz commented 4 years ago

Description of changes:

Motivation for changes: operator-courier is an integral part of manifest verification and pushing to quay.io in OMPS. Every PR can potentially break OMPS, so the desired changes to operator-courier in a PR should be integrated into and tested with the latest OMPS.

Travis CI tests will continue to function as before. These changes are intended to be used in OpenShift CI.

/cc @gallettilance @shawn-hurley @kevinrizza

csomh commented 4 years ago

@estroz small note: OMPS has set of integration test cases defined, and we are running them against a qa instance internally, whenever there is a new release to operator-courier. (They are not part of a CI, b/c we didn't have the time to set that up.)

Would it make sense to use (a subset) of those in OpenShift CI, instead of defining a new (to be maintained) set of test cases?

estroz commented 4 years ago

@csomh if it is possible to trigger an OMPS integration test on each operator-courier PR and report the results in that PR, then we should set a prow CI config up for that instead. AFAIK that won't be possible because prow CI is configured to report on a per-repo basis.

@kevinrizza no, but if ^ is possible then I can create one.

kevinrizza commented 4 years ago

@estroz I'm still not sure I understand how this is supposed to work then. This needs some configuration to run. Where is that configuration defined? Where do these tests actually run? Travis? Did we set those environment variables there?

estroz commented 4 years ago

@kevinrizza these tests will run in prow CI. I will submit a config to openshift/ci-operator once the tests are merged here.

estroz commented 4 years ago

Closing in favor of #164 and follow-up.