Improve error message of signature verification

[yizha1]

What would you like to be added?

The error message of signature verification is not concise and actionable, see example,

Example1: "verification failed: Error: referrers not found, Code: REFERRERS_NOT_FOUND, Component Type: executor"

Example2: "Original Error: (Original Error: (signature is not produced by a trusted signer), Error: verify signature failure, Code: VERIFY_SIGNATURE_FAILURE, Plugin Name: notation, Component Type: verifier, Documentation: https://github.com/notaryproject/notaryproject/tree/main/specs, Detail: failed to verify signature of digest), Error: verify reference failure, Code: VERIFY_REFERENCE_FAILURE, Plugin Name: notation, Component Type: verifier"

It's hard to understand what happened, what happened and what needs to be done. There is a need for enhancements in the error messages related to signature verification. We can test the common scenarios for both Notary Project signatures and Cosign signatures, and check whether error messages are concise, precise, and actionable

• Images are not signed
• Users do not have right roles assigned to access AKV (Notary Project signatures)
• Users do not have right roles assigned to access AKV (Cosign signatures)
• Users cannot access registries
• Images are signed with expected identities/keys that should be able to verify, however, verification configurations are wrong (both Notary Project signatures and Cosign signatures)
• Images are signed by unknown identities so that verification should fail. (Notary Project signatures)
• Images are signed by unknown keys so that verification should fail (Cosign key-pair signatures)
• Images are signed with revoked certificates
• Images are signed before certificates are expired (Time-stamp support is required and users do not specify TSA root certificate)
• Images are signed before certificates are expired (Time-stamp support is required and users specify TSA root certificate)

Anything else you would like to add?

Discussed with Yi offline, we have more scenarios need to be covered, check this doc for more details: https://hackmd.io/@H7a8_rG4SuaKwzu4NLT-9Q/HkFHgokv0#User-Scenarios

Work item break down:

• [ ] KMP configuration
• [ ] Store configuration
• [ ] Verifier configuration
• [ ] Policy configuration
• [ ] Access control to ACR/AKV
• [ ] Signature verification

Are you willing to submit PRs to contribute to this feature?

• [ ] Yes, I am willing to implement it.

[binbin-li]

Scenarios listed in the doc: https://hackmd.io/@H7a8_rG4SuaKwzu4NLT-9Q/HkFHgokv0#User-Scenarios

Work item break down:

• [ ] KMP configuration
• [ ] Store configuration
• [ ] Verifier configuration
• [ ] Policy configuration
• [ ] Access control to ACR/AKV
• [ ] Signature verification

[binbin-li]

Scenarios listed in the doc: https://hackmd.io/@H7a8_rG4SuaKwzu4NLT-9Q/HkFHgokv0#User-Scenarios

Work item break down:

• [ ] KMP configuration
• [ ] Store configuration
• [ ] Verifier configuration
• [ ] Policy configuration
• [ ] Access control to ACR/AKV
• [ ] Signature verification

After discussion with @yizha1, we have proposed some overall improvements to the current error handling framework.

1. Users don't need to know everything about an error(the complete stack trace), they would just need the root cause of the error.
2. We can introduce a new field(reason) to the verifierReports besides message. The message will indicate a general error message, while the reason explaining the root cause.
3. Probably log the error msg at each level instead of logging a wrapped error on the top.
4. Probably add an error field to all CR structs. When an error happens while reconciling, we'll add the error. Then during the artifact validation, executor could fetch the error from reconciling.
5. Consolidate the verifierReports format between configPolicy and regoPolicy. a. Add artifactType to each level. b. Have message and reason fields c. Reduce confusion between type and artifact-type
6. Refactor the nested error structure to make it well formatted while keeping info to users/devs