signing of arbitrary 32 byte digest not working with az keyvault key sign

[mb13sensia]

Describe the bug

It is expected that the digest passed to az keyvault key sign be an arbitrary 32 byte sha-256 checksum.
The example uses only characters 1-9. Passing most valid ascii characters appears to be accepted, however many byte codes cause an exception due to the underlying python trying to convert the input to utf-8

For reference the failing sum (as hex text) is 3650d9b8ccb170c41127ba357e7777fae64f650d1115da5264c36589398f3d07 Example, passing exactly 32 bytes in: az keyvault key sign -a RS256 --digest sha256sum manifest | cut -d " " -f 1| xxd -r -p --id https:// 'utf-8' codec can't encode character '\udcc4' in position 5: surrogates not allowed

equivalent: az keyvault key sign -a RS256 --digest echo 3650d9b8ccb170c41127ba357e7777fae64f650d1115da5264c36589398f3d07 | xxd -r -p --id https://

Attempts to debug where different string lengths are confounded by the system reporting a rest API error about the size of the input to the base64 encoding which is applied by the az tool.

Related command

az keyvault key sign

Errors

'utf-8' codec can't encode character '\udcc4' in position 5: surrogates not allowed

Issue script & Debug output

az keyvault key sign -a RS256 --digest echo 3650d9b8ccb170c41127ba357e7777fae64f650d1115da5264c36589398f3d07 | xxd -r -p --id https://

Expected behavior

A valid response such as when I enter a byte input that doesn't happen to contain illegal utf-8 encoded characters, or support for passing the binary value as hex characters.

Environment Summary

azure-cli 2.53.0

core 2.53.0 telemetry 1.1.0

Dependencies: msal 1.24.0b2 azure-mgmt-resource 23.1.0b2

Python location '/opt/az/bin/python3' Extensions directory '/home/mbrown13/.azure/cliextensions'

Python (Linux) 3.10.10 (main, Sep 20 2023, 06:07:28) [GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

The changing the encoding configuration for this input may be the correct solution .decode('utf8','surrogateescape')

[yonzhan]

Thank you for opening this issue, we will look into it.

[side-chains]

I think it is not clear enough that this issue is a bug.

At this moment, it is impossible to sign anything using the azure cli, unless its hash happens to be text, which is very unlikely.

Additionally, the error message states that the digest value should be encoded with base64url, which is not the case. This would make it work.

[mb13sensia]

This answer is superficial. The fact that the base64 errors are reported incorrectly is well known in the community, these are underlying API errors that are always reported by the backend. Simply try passing a key of the correct size for the selected algorithm with base64 encoding, the CLI will report a error that the input is too large. The only input size it will accept is the exact number of BYTES required for the selected algorithm.

[side-chains]

This answer is superficial. The fact that the base64 errors are reported incorrectly is well known in the community, these are underlying API errors that are always reported by the backend. Simply try passing a key of the correct size for the selected algorithm with base64 encoding, the CLI will report a error that the input is too large. The only input size it will accept is the exact number of BYTES required for the selected algorithm.

Indeed, it is well know that the error message is wrong, and it is also well know that the command line expects the bytes of the hash, rather than their hex encoding. However, it is not possible to pass arbitrary bytes to as a bash command line argument, which is what makes this issue a :bug:.

[AliakseiT]

To the best of my understanding, without the fix for this issue the only reasonable way to use key vault keys is to download the private part, use it, and then immediately discard. Which, diminishes the value of key vault as such.

[side-chains]

To the best of my understanding, without the fix for this issue the only reasonable way to use key vault keys is to download the private part, use it, and then immediately discard. Which, diminishes the value of key vault as such.

You can use the rest api:

curl -s -X POST -H "Authorization: Bearer ${BEARER}" -H "Content-Type:application/json" -H "Accept:application/json" "https://da-vault.vault.azure.net/keys/da-key/01234567890123456789012345678901/sign?api-version=7.4" -d '{ "alg" : "ES256", "value" : "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=" }'

[AliakseiT]

To the best of my understanding, without the fix for this issue the only reasonable way to use key vault keys is to download the private part, use it, and then immediately discard. Which, diminishes the value of key vault as such.

You can use the rest api:

curl -s -X POST -H "Authorization: Bearer ${BEARER}" -H "Content-Type:application/json" -H "Accept:application/json" "https://da-vault.vault.azure.net/keys/da-key/01234567890123456789012345678901/sign?api-version=7.4" -d '{ "alg" : "ES256", "value" : "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=" }'

That worked, thanks!

[mb13sensia]

My Organization has taken a similar (and agreed, less secure) approach and just given up on using an actual key-vault keys. We are just storing key pairs as base64 encoded secrets in key-vault and write our own scripts to actually handle the encoding/decoding properly. This also gives us the flexibility to move to a entirely different service provider easily.

[sburton84]

It's very strange that the CLI seems to expect raw bytes on the command-line instead of a printable encoding such as hex or base64, it would make much more sense to just be able to pass a string that is already base64 (or hex) encoded.

[Sifungurux]

@evelyn-ys any update on this bug. The functionality of keyvault as a kms is now at the moment useless

[freedge]

You can use the rest api:

the rest api worked for me for RSA keys but not for EC keys so far

[richardfan1126]

@evelyn-ys any update on this bug?

I can't believe a cli tool expecting a raw byte input