Bug: Panic when verifying policy, error "unknown variant `ecdsa`"

[viccuad]

When running kwctl verify as follows, mirroring a failure shown on integration tests in CI, I get:

2024-03-20T10:36:23.248610Z  WARN kwctl: Cannot fetch TUF repository: TufError(ParseMetadata { role: Root, source: Error("unknown variant `ecdsa`, expected one of `rsa`, `ed25519`, `ecdsa-sha2-nistp256`", line: 9, column: 22), backtrace: Backtrace(()) })

This happens for test policy registry://ghcr.io/kubewarden/tests/capabilities-psp:v0.1.9and GA registry://ghcr.io/kubewarden/policies/capabilities-psp:v0.1.15

Full output:

Click me

```console $ kwctl --version kwctl 1.11.0-rc5 Use the `info` command to display system information. $ kwctl verify --verification-config-path verification-config.yml registry://ghcr.io/kubewarden/tests/capabilities-psp:v0.1.9 2024-03-20T10:36:23.248610Z WARN kwctl: Cannot fetch TUF repository: TufError(ParseMetadata { role: Root, source: Error("unknown variant `ecdsa`, expected one of `rsa`, `ed25519`, `ecdsa-sha2-nistp256`", line: 9, column: 22), backtrace: Backtrace(()) }) 2024-03-20T10:36:23.248644Z WARN policy_fetcher::verify: Sigstore Verifier created without Fulcio data: keyless signatures are going to be discarded because they cannot be verified 2024-03-20T10:36:23.248649Z WARN policy_fetcher::verify: Sigstore Verifier created without Rekor data: transparency log data won't be used 2024-03-20T10:36:23.248652Z WARN policy_fetcher::verify: Sigstore capabilities are going to be limited 2024-03-20T10:36:23.248656Z INFO sigstore::cosign::client_builder: Rekor public key not provided. Rekor integration disabled 2024-03-20T10:36:23.248660Z INFO sigstore::cosign::client_builder: No Fulcio cert has been provided. Fulcio integration disabled 2024-03-20T10:36:25.214557Z INFO sigstore::cosign::signature_layers: Ignoring bundle, rekor public key not provided to verification client bundle="{\"SignedEntryTimestamp\":\"MEUCIQCTsSmCHAnPYzjENHAUceoUqqjzNYyamfmXfmtjwszsDwIgHxzHN5c476RJZfS5xy6k7e1NiwUvqkr2w+Kd9u6/f2c=\",\"Payload\":{\"body\":\"eyJhcGlWZXJzaW9uIjoiMC4wLjEiLCJraW5kIjoiaGFzaGVkcmVrb3JkIiwic3BlYyI6eyJkYXRhIjp7Imhhc2giOnsiYWxnb3JpdGhtIjoic2hhMjU2IiwidmFsdWUiOiIwODFlZDJjYzJhYTZhYjA1YTQ0MzFkNDM1ZTNkNWViOGVhMzFjYTJjNTZiNjQwNDM4NGQxMjRjMmMwYWNkYzcxIn19LCJzaWduYXR1cmUiOnsiY29udGVudCI6Ik1FVUNJQk5JQ1Q3ZGtoVkNHd3ZyTk45RlZPRVhQWUxyWVA0elRaSGpqVHRVazVZTkFpRUE1SkxmS1NSK05XUUZlUytXbjE3aFl3eTNML2lFSFpod0dNS3AzV1VkSWxvPSIsInB1YmxpY0tleSI6eyJjb250ZW50IjoiTFMwdExTMUNSVWRKVGlCRFJWSlVTVVpKUTBGVVJTMHRMUzB0Q2sxSlNVUlRWRU5EUVhNclowRjNTVUpCWjBsVlFVNUZlVzR4WjJKWVFVdFJVR0ZYVW5CUVIxTmxNMDVGTjIxbmQwTm5XVWxMYjFwSmVtb3dSVUYzVFhjS1MycEZWazFDVFVkQk1WVkZRMmhOVFdNeWJHNWpNMUoyWTIxVmRWcEhWakpOVWtWM1JIZFpSRlpSVVVSRmQyaDZZVmRrZW1SSE9YbGFWRUZsUm5jd2VRcE5ha0Y0VFdwamVFNTZVWGROZW14aFJuY3dlVTFxUVhoTmFtTjRUbnBWZDAxNmFHRk5RazE0UlZSQlVFSm5UbFpDUVc5VVEwaE9jRm96VGpCaU0wcHNDazFHYTNkRmQxbElTMjlhU1hwcU1FTkJVVmxKUzI5YVNYcHFNRVJCVVdORVVXZEJSVEZxU0RKMmJtbHlkVkYxUzFNMWFFZHJkSGhSZFRVdmVWVlRaM2dLWTFreVEyc3pXR2hOVFRKdlVWSnhObVJPZWsxaE5GaERZVFpFUW1oWWVTdGFURkpUVURJMGFqSmhlVkY1WjNaWU5IRlBVRnBNYUVaNFlVOURRV1ZuZHdwblowaHJUVUUwUjBFeFZXUkVkMFZDTDNkUlJVRjNTVWhuUkVGVVFtZE9Wa2hUVlVWRVJFRkxRbWRuY2tKblJVWkNVV05FUVhwQlRVSm5UbFpJVWsxQ0NrRm1PRVZCYWtGQlRVSXdSMEV4VldSRVoxRlhRa0pSWTNaQlNsbFZaM293WmxWSUwxY3JTMlkwYkVaR1pETXJSRWhxUVdaQ1owNVdTRk5OUlVkRVFWY0taMEpTV1hkQ05XWnJWVmRzV25Gc05ucEtRMmhyZVV4UlMzTllSaXRxUWpSQ1owNVdTRkpGUldOVVFuWm9iVEZ2WkVoU2QyTjZiM1pNTW1Sd1pFZG9NUXBaYVRWcVlqSXdkbUV6Vm1sYVdHUm9ZMjFTYkdKcE9XNWhXRkp2WkZkSmRGbFhUakJoVnpsMVkzazRkVm95YkRCaFNGWnBURE5rZG1OdGRHMWlSemt6Q21ONU9YbGFXRlo2V1ZkS2MxcFRNWGxhVjNoc1dWaE9iRXhZUW5aaVIyeHFaVk14ZVdSWVRqQk1ibXgwWWtWQ2VWcFhXbnBNTW1oc1dWZFNla3d6V1hnS1RVSTBSME5wYzBkQlVWRkNaemM0ZDBGUldVVkZTRXBzV201TmRtUkhSbTVqZVRreVRVTTBlRXhxYTNkRloxbExTM2RaUWtKQlIwUjJla0ZDUVdkUlJRcGpTRlo2WVVSQlkwSm5iM0pDWjBWRlFWbFBMMDFCUlVWQ1FUVlRXbGQ0YkZsWVRteEpTRUoyWWtkc2FtVlVRVEpDWjI5eVFtZEZSVUZaVHk5TlFVVkVDa0pEYUdsTmFteG9Xa2RKTVZscVdUTlBWMVpwVGtSU2FFOVhSbXRaTWxFeFdUSlplazVxVVhkYVZFMDFXa2RXYTAxVWEzZFplbGw1VFVSclIwTnBjMGNLUVZGUlFtYzNPSGRCVVVWRlN6Sm9NR1JJUW5wUGFUaDJaRWM1Y2xwWE5IVlpWMDR3WVZjNWRXTjVOVzVoV0ZKdlpGZEtNV015Vm5sWk1qbDFaRWRXZFFwa1F6VnFZakl3ZDAxQldVdExkMWxDUWtGSFJIWjZRVUpDVVZGcFlUTldhVnBZWkdoamJWSnNZbWs1YWxsWVFtaFpiV3h6WVZoU2NGcFlUWFJqU0U1M0NreFlRblppUjJ4cVpWUkJTMEpuWjNGb2EycFBVRkZSUkVGM1RtOUJSRUpzUVdwRlFXeG9ia0UzTDJGWGVVaGtTMFEwY1ZKbWRHWm5hRmh4U1ZoQ1NWWUtaV2d5VFZWTWFrOU5aRVpLWmxOU2MzSjVPWGRpZVdsNlFpdHRlWGgxZG1SMFpEaE1RV3BDWVhsS2JVY3pLekYzT0VOcE9HVldOMFZoWmtWUVZ6TnlNd3BPT1VsSlJ6QnZTVmhTYjNRMlRWazNWVTVUUTNsQ2NHTmFTMUJqTXpZd2VsbGxPSG8xTjJzOUNpMHRMUzB0UlU1RUlFTkZVbFJKUmtsRFFWUkZMUzB0TFMwSyJ9fX19\",\"integratedTime\":1643305239,\"logIndex\":1181814,\"logID\":\"c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d\"}}" 2024-03-20T10:36:25.214765Z INFO sigstore::cosign::signature_layers: Ignoring certificate annotation reason="fulcio certificates not provided" Error: Policy registry://ghcr.io/kubewarden/tests/capabilities-psp:v0.1.9 cannot be validated Image verification failed: missing signatures The following constraints were not satisfied: kind: githubAction owner: kubewarden repo: null annotations: null Stack backtrace: 0: anyhow::error:: for anyhow::Error>::from 1: kwctl::verify::verify::{{closure}} 2: kwctl::main::{{closure}} 3: tokio::runtime::park::CachedParkThread::block_on 4: tokio::runtime::runtime::Runtime::block_on 5: kwctl::main 6: std::sys_common::backtrace::__rust_begin_short_backtrace 7: std::rt::lang_start::{{closure}} 8: core::ops::function::impls:: for &F>::call_once at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13 9: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 10: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 11: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 12: std::rt::lang_start_internal::{{closure}} at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48 13: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 14: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 15: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 16: std::rt::lang_start_internal at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20 17: main Stack backtrace: 0: anyhow::error::::msg 1: kwctl::main::{{closure}} 2: tokio::runtime::park::CachedParkThread::block_on 3: tokio::runtime::runtime::Runtime::block_on 4: kwctl::main 5: std::sys_common::backtrace::__rust_begin_short_backtrace 6: std::rt::lang_start::{{closure}} 7: core::ops::function::impls:: for &F>::call_once at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13 8: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 9: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 10: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 11: std::rt::lang_start_internal::{{closure}} at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48 12: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 13: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 14: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 15: std::rt::lang_start_internal at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20 16: main ```

The verification-config.yml is straight from kwctl scaffold verification-config > verification-config.yml:

# Default Kubewarden verification config
#
# With this config, the only valid policies are those signed by Kubewarden
# infrastructure.
#
# This config can be saved to its default location (for this OS) with:
#   kwctl scaffold verification-config > /home/vic/.config/kubewarden/verification-config.yml
#
# Providing a config in the default location enables Sigstore verification.
# See https://docs.kubewarden.io for more Sigstore verification options.
apiVersion: v1
allOf:
- kind: githubAction
  owner: kubewarden
  repo: null
  annotations: null
anyOf: null

Note:

While this makes image verification fail in kwctl and policy-server, we fail-closed, meaning that even if images are correctly signed, Kubewarden will fail the image verification and report "Image verification failed: missing signatures".

[flavio]

Seems to be caused by https://github.com/sigstore/sigstore-rs/issues/338

[viccuad]

This needs: a. Patched awslabs/touch: https://github.com/viccuad/tough/tree/patch-edcsa, see compare

b. Either use a [patch.crates-io] approach:

c. Or consume manually:

1. Patched sigstore-rs: https://github.com/viccuad/sigstore-rs/tree/patch-edcs see compare
2. Patched policy-fetcher
3. Patched policy-evaluator

d. Releases for kwctl and policy-server.

[viccuad]

Trying approach b, [patch.crates-io] is not reasonable.

The patch.crates.io Rust feature doesn't support crate features nor default features. We need to consume awslabs/touch with the http feature enabled. One can't set it as default, hence one needs to remove the feature and hardcode it (same for awslabs/touch, awslabs/tough-ssm as they expect that http feature) (tried here). Still, doing that means patching sigstore-rs to not consume awslabs/tought with the feature, and that brings the same problem again.

Simpler to fork the dependency chain, approach c.

[viccuad]

Consumed the pointed forks above in policy-fetcher, policy-evaluator, kwctl, and the upstream tough fix seems to be incomplete.

Moving to blocked.