search

sigstore / sigstore-conformance

Conformance testing for Sigstore clients
https://sigstore.dev
7 stars 10 forks source link

unexpected success should lead to printing args #107

Closed jku closed 11 months ago

jku commented 11 months ago

If the client fails unexpectedly there's comprehensive debug output from SigstoreClient:

_CLIENT_ERROR_MSG = """
!!! CLIENT FAILURE !!!

Arguments: {args}
Exit code: {exitcode}

!!! STDOUT !!!
==============

{stdout}

!!! STDERR !!!
==============

{stderr}
"""

but in the case of client succeeding unexpectedly (with pytest.raises()) you only get

E       Failed: DID NOT RAISE <class 'test.client.ClientFail'>

This should really print at least the arguments (but for completeness maybe also stdout and stderr) to make it easier for client developers to reproduce.

A custom context manager like SigstoreClient.raises() might work.