sigstore / sigstore-conformance

Conformance testing for Sigstore clients
https://sigstore.dev
7 stars 10 forks source link

Proposal: add "test vector" tests to this repo. #64

Open znewman01 opened 1 year ago

znewman01 commented 1 year ago

Previously, we've discussed the merits of two sorts of conformance testing:

I, for one, left that conversation convinced that it's a good idea to do both.

We're starting to pick up on test vector work (e.g., https://github.com/sigstore/protobuf-specs/issues/66 ; CC @asraa). At the Sigstore clients meeting the other day (CC @bdehamer, @codysoyland, @kommendorkapten) we discussed where these (and related harness code etc.) should live. While previously, we had planned to put them in sigstore/protobuf-specs, it seems like sigstore/conformance-tests is a quite natural fit.

If you all agree, we can add those in here, update the README to articulate the overall test strategy, and use this as the home for test vector testing. If you think they'd be better off elsewhere, we can discuss where too.

woodruffw commented 1 year ago

That works for me! I agree that we want both kinds of conformance testing, and a singular point of reference for them makes sense to me.

tetsuo-cpp commented 1 year ago

Sounds good. I also agree this seems like a good spot for it.