Closed jku closed 11 months ago
An unexpected success now looks like this:
_____________________________ test_verify_rejects_invalid_inclusion_proof ______________________________ client = <test.client.SigstoreClient object at 0x7f842a9066d0> make_materials_by_type = <function make_materials_by_type.<locals>._make_materials_by_type at 0x7f842a90c0e0> def test_verify_rejects_invalid_inclusion_proof( client: SigstoreClient, make_materials_by_type: _MakeMaterialsByType ) -> None: """ Check that the client rejects a bundle with an old inclusion proof """ materials: BundleMaterials input_path, materials = make_materials_by_type("a.txt", BundleMaterials) materials.bundle = Path("a.txt.invalid_inclusion_proof.sigstore") > with client.raises(): /home/jkukkonen/src/sigstore-conformance/test/test_bundle.py:142: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.11/contextlib.py:144: in __exit__ next(self.gen) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <test.client.SigstoreClient object at 0x7f842a9066d0> @contextmanager def raises(self): try: yield except ClientFail: pass else: assert self.completed_process msg = _CLIENT_ERROR_MSG.format( exitcode=self.completed_process.returncode, command=" ".join(map(str, self.completed_process.args)), stdout=self.completed_process.stdout, stderr=self.completed_process.stderr, ) > raise ClientUnexpectedSuccess(msg) E test.client.ClientUnexpectedSuccess: E Command: /home/jkukkonen/src/sigstore-conformance/sigstore-python-conformance verify-bundle --bundle a.txt.invalid_inclusion_proof.sigstore --certificate-identity https://github.com/sigstore-conformance/extremely-dangerous-public-oidc-beacon/.github/workflows/extremely-dangerous-oidc-beacon.yml@refs/heads/main --certificate-oidc-issuer https://token.actions.githubusercontent.com a.txt E Exit code: 0 E E !!! STDOUT !!! E ============== E E OK: a.txt E E E !!! STDERR !!! E ============== E E None /home/jkukkonen/src/sigstore-conformance/test/client.py:158: ClientUnexpectedSuccess
It would be even nicer if the custom raises() wasn't shown in the stack trace but I think this is an improvement:
I removed the "CLIENT FAILURE" line from output (the exception name should be very clear already: ClientFail or ClientUnexpectedSuccess).
Fixes #107
Summary / Release note
An unexpected success now looks like this:
It would be even nicer if the custom raises() wasn't shown in the stack trace but I think this is an improvement:
I removed the "CLIENT FAILURE" line from output (the exception name should be very clear already: ClientFail or ClientUnexpectedSuccess).
Fixes #107