sigstore / sigstore-conformance

Conformance testing for Sigstore clients
https://sigstore.dev
7 stars 10 forks source link

Tweak staging invocation #131

Closed jku closed 6 months ago

jku commented 6 months ago

Main change here is the staging invocation: I'm proposing that a single action invocation only runs the testsuite once because

Other changes in this PR (from testing and reviewing for a hopefully near future release):

Fixes #130

I'll mark William a reviewer since we originally agreed to run test suite twice but I think we may have been wrong.


For release Notes:

woodruffw commented 6 months ago

I'm proposing that a single action invocation only runs the testsuite once because

  • xfails becames painful if staging expected failures are not the same as prod (which seems like a normal thing at times).
  • The test results are much clearer ("Staging conformance failure" is not as critical as "production conformance failure")

Makes sense to me, thanks for documenting!