Description of issue or feature request:

https://github.com/secure-systems-lab/securesystemslib/issues/251 discusses various consistency concerns related to securesystemslib (TUF/in-toto) cryptographic keys.

This is an attempt to carve (and flesh) out only those concerns related to the format of public keys as they appear in TUF/in-toto metadata, with the goal of a clear specification likely as part (or appendix) of the new signing-spec and its reference implementation in securesystemslib.

Below in 'Current behavior' is an exhaustive list of available values for public keys as currently implemented in securesystemslib. This includes rsa, ecdsa, ed25519 and spx (WIP #169) keys generated with securesystemslib, plus keys exported from a GnuPG keyring using securesystemslib.gpg or from a cryptographic token (e.g. HSM) using securesystemslib.hsm (WIP #229).

Related concerns and calls for action are listed further below in 'Expected behavior'.

Current behavior:

Public key container (mandatory fields)

{
  'keyid' <...>, # Optional in the spec, but included in the in-toto reference implementation
  'keytype': <...>,
  'scheme': <...>,
  'keyval': {
    'private': <...>, # Must be omitted or empty for public keys
    'public': <...>
  }
}

'keyid'

ascii hex of canonical JSON (OLPC) of parts of the key dictionary
RFC4880 OpenPGP Issuer Fingerprint (gpg keys only)
custom value (see WIP #169 , TAP 12 and https://github.com/theupdateframework/tuf/issues/1084)

Note that 'keyid' is not part of the public key metadata format according to in-toto/TUF specifications and tuf reference implementation, but it is included in the in-toto reference implementation (see #251 PR description).

'keytype'

'rsa'
'ed25519'
'ecdsa'
'ecdsa-sha2-nistp(256|384)' (legacy, new keys use 'ecdsa', see #239 and #267)
'spx' (WIP #169)

'scheme'

'rsassa-pss-(md5|sha1|sha224|sha256|sha384|sha512)'
'rsa-pkcs1v15-(md5|sha1|sha224|sha256|sha384|sha512)'
'ecdsa-sha2-nistp(256|384)'
'ed25519'
'spx' (WIP #169)

'type'

(gpg keys only, instead of 'keytype')

'rsa'
'dsa'
'eddsa'

'method'

(gpg keys only, instead of 'scheme')

'pgp+rsa-pkcsv1.5'
'pgp+dsa-fips-180-2'
'pgp+eddsa-ed25519'

'hashes'

(gpg keys only, instead of 'scheme')

'pgp+SHA2'

'keyval'

'public'

rsa, ecdsa, ecdsa from HSM (WIP #229) X.509 SubjectPublicKeyInfo PEM string (RFC5280)
ed25519 32 bytes ascii hex key data (RFC8032)
spx (WIP #169) 64 bytes key data (SPHINCS+-SHAKE256)

rsa gpg (RFC4880)

{
  "e": <public encryption exponent (ascii hex)>,
  "n": <public modulus (ascii hex)>
}

dsa gpg (RFC4880)

{
"y": <public value (ascii hex)>,
"p": <public modulus (ascii hex)>,
"q": <group order (ascii hex)>,
"g": <group generator (ascii hex)>
}

eddsa gpg (RFC4880)

{
"q": <MPI of an EC point representing a public key (ascii hex)>
}

'private'

(NOTE: The 'private' portion is not directly relevant for this issue, as it must not be included in TUF/in-toto metadata. For the sake of completeness we still list the private key values that are currently kept internally in securesystemslib).

rsa, ecdsa PKCS#1/TraditionalOpenSSL PEM string (RFC8017)
ed25519 32 bytes ascii hex key data (RFC8032)
spx (WIP #169) 128 bytes key data (SPHINCS+-SHAKE256)**

Optional fields

'creation_time'
'validity_period'
'expires'
'keyid_hash_algorithms' (https://github.com/theupdateframework/tuf/issues/1084)

Expected behavior:

'keyid'
- better decouple default keyids from keys, to allow the use of custom keyids, but with a canonical json computed keyid as fallback
- clarify if keyid is optional in the key object
Consolidate 'keytype', 'type', 'scheme', 'method', 'hashes' fields and values
- 'scheme' usually includes key type, signing scheme and hash algorithm, could one field be enough?
- revise correctness/appropriateness of values (see e.g. 'eddsa' vs. 'ed25519' as different '(key)type's for the same type of key).
- is there a standard for 'scheme' strings that we could use?
'keyval'
- consolidate values (see e.g. serialized rsa PEM vs. non-serialized rsa numbers)
- clarify if 'private' field may be omitted or should be just empty for public keys
optional fields
- deprecate 'keyid_hash_algorithms'
- maybe consolidate 'validity_period' and 'expires'?

It's hard to tell when to mark this issue as resolved, but I would say that the public key metadata format has been revised. Realistically, there won't be many changes, given that past changes have been quite disruptive for users.

The good news is that the implementation is way better decoupled from the public key metadata format these days, which makes all of this easier to address.

I suggest to fix remaining concerns above all in docs, for which we have more reasonably scoped tickets. See inline...

Expected behavior:

'keyid'

better decouple default keyids from keys, to allow the use of custom keyids, but with a canonical json computed keyid as fallback

done

clarify if keyid is optional in the key object

The Key interfaces requires a keyid, the default serialization methods (to/from_dict), however, do not include a keyid. Users can easily override this behavior.

Related issues: #416, #670

Consolidate 'keytype', 'type', 'scheme', 'method', 'hashes' fields and values

The concerns about redundancy and semantic correctness of keytype/scheme pairs are still valid, however, changing the existing values is not worth the effort (see all the issues related to the ecdsa keytype change).

method and hashes are specific to the securesystemslib.gpg subpackage, which is not part of the documented API. GPGKey provides a compatibility layer, which uses keytype/scheme instead.

'keyval'

consolidate values (see e.g. serialized rsa PEM vs. non-serialized rsa numbers)

The exact keyval format is a detail of a specific Key implementation. Despite inconsistency, the current formats for existing keytype/scheme pairs are unlikely to change for the reasons described above.

clarify if 'private' field may be omitted or should be just empty for public keys

Private keys are now separated (Signer) from public keys (Key). This means that securesystemlsib no longer populates or uses a private field.

optional fields

The Key interface supports de/serialization of unrecognised fields.

Related issues: #467, #587

deprecate 'keyid_hash_algorithms'