Support loading cryptographic keys from files other than certificate PFX

[yaakov-h]

I understand that this represents a thin wrapper over the Win32 CNG APIs, but it does make a whole suite of cryptographic operations simpler.

In my case, I'm trying to read an ECDSA public/private key pair out of a PKCS8 container. In .NET Framework and .NET Core on Windows, this is possible with the following:

static ECDsa CreateKeyFromDataCng(byte[] data)
    => new ECDsaCng(CngKey.Import(data, CngKeyBlobFormat.Pkcs8PrivateBlob));

Under macOS (and I assume Linux too), this throws a PlatformNotSupportedException.

As per dotnet/runtime#21259, .NET Core can handle loading ECDSA keys from an X509 structure.

Would it be possible to implement the Cng suite of APIs with a backing from a non-Windows-CNG source such as OpenSSL on Linux or the Apple crypto libraries on macOS?

Failing that, are there any plans for a unified cryptography API in .NET Standard future which would enable a single API call to work across all platforms for fiddly cryptography bits such as this?

Thanks.

---

Feature Proposal

Add API which allows binary-encoded asymmetric cryptographic keys from standard data formats to be loaded, and add API which allows asymmetric cryptographic keys to be exported to those same standard data formats (subject to key exportability permissions).

Based on competitive landscape and other research, the following data formats are to be included:

• Public Key (generic)
  • X.509 SubjectPublicKeyInfo (RFC 3280, ITU-T REC X.509, et al).
• Private Key (generic)
  • PKCS#8 PrivateKeyInfo (IETF RFC 5208)
  • PKCS#8 EncryptedPrivateKeyInfo (IETF RFC 5208)
  • Reading data protected with PBES1, PBES2, and PKCS12-PBE.
  • Writing data protected with PBES2 and subsets of PKCS12-PBE.
• Algorithm-Specific
  • RSA
  • RSAPublicKey (PKCS#1, IETF RFC 3447, et al)
  • RSAPrivateKey (PKCS#1, IETF RFC 3447, et al)
  • DSA
    • (N/A)
  • ECDsa
  • ECPrivateKey (SECG SEC1v2)
  • ECDiffieHellman
  • ECPrivateKey (SECG SEC1v2)

Notably, the ECPublicKey type does not appear in this listing due to not finding standards which transported it without the SubjectPublicKeyInfo wrapper.

Caveats: On Linux, in particular, PEM-encoding (from the Privacy Enhanced Mail specification) is more standard. PEM, as a data format, can contain extra metadata in addition to the BER-encoded payload. Rather than add PEM-reading complications into these APIs the recommendation is to build a separate PEM reading class, which is capable of reporting on the additional metadata.

The data export methods are repeated in the API proposal, once to write to a destination Span<byte> (TryExport*) and again to return a byte[] (Export*).

Rather than do content-sniffing, these APIs are explicit as to their data format.

PKCS#8 EncryptedPrivateKeyInfo methods have two variants:

• A char-based variant will generally convert the textual input to UTF-8 bytes, but PKCS12-PBE operates on the textual input as UTF16-BE (as per the specification).
• A byte-based variant will generally accept the input bytes as the input to the Key Derivation Function (KDF) dictated by the encryption options. Since this is not valid for PKCS12-PBE, this variant will throw when the input data was protected with PKCS12-PBE.

override strategy

PKCS#8 data formats can contain additional metadata (attributes). Derived types using implementations which respect those attributes (such as Windows CNG) should defer to their underlying platform's PKCS#8 importer and exporter. Otherwise, the overrides (or initial definitions) on the algorithm-specific base classes should be sufficient (which call the existing ImportParameters and ExportParameters routines).

Power scenario

To allow for the creation of PKCS#8 data containing custom attributes (as well as reading those attributes), a class for reading and writing PKCS#8 will also be added. In this type the default will be to make defensive copies of inputs, but options are exposed to reuse input memory when possible.

API Proposal

System.Security.Cryptography.Primitives:

namespace System.Security.Cryptography
{
    // X.509 SubjectPublicKeyInfo
    public abstract partial class AsymmetricAlgorithm : System.IDisposable
    {
        public virtual void ImportSubjectPublicKeyInfo(ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual byte[] ExportSubjectPublicKeyInfo() => throw null;
        public virtual bool TryExportSubjectPublicKeyInfo(Span<byte> destination, out int bytesWritten) => throw null;
    }

    // PKCS#8 PrivateKeyInfo
    public abstract partial class AsymmetricAlgorithm : System.IDisposable
    {
        public virtual void ImportPkcs8PrivateKey(ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual byte[] ExportPkcs8PrivateKey() => throw null;
        public virtual bool TryExportPkcs8PrivateKey(Span<byte> destination, out int bytesWritten) => throw null;
    }

    // PKCS#8 EncryptedPrivateKeyInfo
    // PBE: Password-Based Encryption (PKCS#5, IETF RFC 2898)
    public enum PbeEncryptionAlgorithm
    {
        Unknown = 0,
        Aes128Cbc = 1,
        Aes192Cbc = 2,
        Aes256Cbc = 3,
        TripleDes3KeyPkcs12 = 4,
    }
    public sealed class PbeParameters
    {
        public PbeEncryptionAlgorithm EncryptionAlgorithm { get; }
        public HashAlgorithmName HashAlgorithm { get; }
        public int KdfIterationCount { get; }
        public PbeParameters(PbeEncryptionAlgorithm encryptionAlgorithm, HashAlgorithmName hashAlgorithm, int kdfIterationCount) { }
    }
    public abstract partial class AsymmetricAlgorithm : System.IDisposable
    {
        public virtual void ImportEncryptedPkcs8PrivateKey(ReadOnlySpan<byte> passwordBytes, ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual void ImportEncryptedPkcs8PrivateKey(ReadOnlySpan<char> password, ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual byte[] ExportEncryptedPkcs8PrivateKey(ReadOnlySpan<byte> passwordBytes, PbeParameters pbeParameters) => throw null;
        public virtual byte[] ExportEncryptedPkcs8PrivateKey(ReadOnlySpan<char> password, PbeParameters pbeParameters) => throw null;
        public virtual bool TryExportEncryptedPkcs8PrivateKey(ReadOnlySpan<byte> passwordBytes, PbeParameters pbeParameters, Span<byte> destination, out int bytesWritten) => throw null;
        public virtual bool TryExportEncryptedPkcs8PrivateKey(ReadOnlySpan<char> password, PbeParameters pbeParameters, Span<byte> destination, out int bytesWritten) => throw null;
    }
}

System.Security.Cryptography.Algorithms (override methods omitted)

namespace System.Security.Cryptography
{
    public abstract partial class RSA : System.Security.Cryptography.AsymmetricAlgorithm
    {
        public virtual void ImportRSAPrivateKey(ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual void ImportRSAPublicKey(ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual byte[] ExportRSAPrivateKey() => throw null;
        public virtual byte[] ExportRSAPublicKey() => throw null;
        public virtual bool TryExportRSAPrivateKey(Span<byte> destination, out int bytesWritten) => throw null;
        public virtual bool TryExportRSAPublicKey(Span<byte> destination, out int bytesWritten) => throw null;
    }
    public abstract partial class ECDiffieHellman : System.Security.Cryptography.AsymmetricAlgorithm
    {
        public virtual void ImportECPrivateKey(ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual byte[] ExportECPrivateKey() => throw null;
        public virtual bool TryExportECPrivateKey(Span<byte> destination, out int bytesWritten) => throw null;
    }
    public abstract partial class ECDsa : System.Security.Cryptography.AsymmetricAlgorithm
    {
        public virtual void ImportECPrivateKey(ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public virtual byte[] ExportECPrivateKey() => throw null;
        public virtual bool TryExportECPrivateKey(Span<byte> destination, out int bytesWritten) => throw null;
    }
}

System.Security.Cryptography.Pkcs

namespace System.Security.Cryptography.Pkcs
{
    public sealed partial class Pkcs8PrivateKeyInfo
    {
        public Oid AlgorithmId { get; }
        public ReadOnlyMemory<byte>? AlgorithmParameters { get; }
        public CryptographicAttributeObjectCollection Attributes { get; }
        public ReadOnlyMemory<byte> PrivateKeyBytes { get; }
        public Pkcs8PrivateKeyInfo(Oid algorithmId, ReadOnlyMemory<byte>? algorithmParameters, ReadOnlyMemory<byte> privateKey, bool skipCopies = false) { }
        public static Pkcs8PrivateKeyInfo Create(AsymmetricAlgorithm privateKey) => throw null;
        public static Pkcs8PrivateKeyInfo Decode(ReadOnlyMemory<byte> source, out int bytesRead, bool skipCopy = false) => throw null;
        public byte[] Encode() => throw null;
        public byte[] Encrypt(ReadOnlySpan<char> password, PbeParameters pbeParameters) => throw null;
        public byte[] Encrypt(ReadOnlySpan<byte> passwordBytes, PbeParameters pbeParameters) => throw null;
        public bool TryEncode(Span<byte> destination, out int bytesWritten) => throw null;
        public bool TryEncrypt(ReadOnlySpan<char> password, PbeParameters pbeParameters, Span<byte> destination, out int bytesWritten) => throw null;
        public bool TryEncrypt(ReadOnlySpan<byte> passwordBytes, PbeParameters pbeParameters, Span<byte> destination, out int bytesWritten) => throw null;
        public static Pkcs8PrivateKeyInfo DecryptAndDecode(ReadOnlySpan<char> password, ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
        public static Pkcs8PrivateKeyInfo DecryptAndDecode(ReadOnlySpan<byte> passwordBytes, ReadOnlyMemory<byte> source, out int bytesRead) => throw null;
    }
}

[bartonjs]

@yaakov-h Are you looking for

• X-Plat CNG (which is a misnomer)
• The ability to load "a key" without knowing what it is (a la the CngKey type)
• The ability to load a PKCS#8 blob as a typed key (e.g. ECParameters.FromPkcs8(data))

I don't know if there's an issue for the last one specifically, but it's on my radar as something important. I feel like that would solve your problems here without making things murky by having another family of classes which partially work x-plat.

[yaakov-h]

In a perfect world I'd have the ability to load an arbitrary key from a standard PEM/DER. In practice for my particular use case I need an ECDsa object to create an ECDsaSecurityKey object for the Microsoft.IdentityModel JWT libraries. This has probably been driven by the way the current underlying crypto APIs are structured.

In this case, the ability to load a known type from a known container (PKCS8 to ECDsa) would suffice. ECParameters.FromPkcs8Data isn't a perfect solution - users with PEMs will still have to parse the -----BEGIN PRIVATE KEY----- container manually - but it's usable.

[bartonjs]

users with PEMs will still have to parse the -----BEGIN PRIVATE KEY----- container manually

Not necessarily... The X509Certificate/2 constructors accept both PEM and DER encoded data, nothing says that a load-from-parameters method for PKCS#8 and SubjectPublicKeyInfo would be DER-only.

[yaakov-h]

True, I just assumed that From...Data would mean from the actual PKCS8 data, which is the base64-encoded blob inside the BEGIN/END lines. If we can handle the textual container too in corefx, then 👍 👍

[danmoseley]

@bartonjs estimates 1 week remaining

[KLuuKer]

Hi I would also really like some way to "easily" import and export RSA & ECDsa, public & private keys directly to and from PEM & DER files. When working with payment providers & certain more exotic external api's require me todo this on a regular basis.

And I end up writing allot of (hopefully error free) code that does the importing and exporting.

Bonus points for allowing cryptographic operations directly on the public&private keys as I sometimes need to pass data trough a private key to "sign" the data, or pass it trough the public key to "encrypt" the data, and I cannot change that because the external api's are from other companies that just set it up that way.

[KLuuKer]

I see the PEM\DER reader\writer is tracked on dotnet/corefx#21833 Only thing we then need is a "small" (probably lot's of code) helper method that does the actual import\exporting. But please don't require a certificate and just let us import\export the keys directly!

[bartonjs]

Bonus points for allowing cryptographic operations directly on the public&private keys as I sometimes need to pass data trough a private key to "sign" the data, or pass it trough the public key to "encrypt" the data,

I'm a little confused by this, since "directly on the public&private keys" is where the operations are already defined...

But please don't require a certificate and just let us import\export the keys directly!

Yep, that's the purpose of this feature :smile:. I think I'm close to happy with the shape of dependencies and data formats in local development, so this should end up with an API proposal relatively soon.

[KLuuKer]

"directly on the public&private keys" as in I take my raw byte[] of data and RSA them trough either the public OR private key and vice versa, yeah some api's require me todo this and I always end up taking dependency on bouncycastle code

[bartonjs]

@KLuuKer I'm still not following.

The RSA and ECDsa classes has SignData, VerifyData, SignHash, VerifyHash (DSA called them SignData, VerifyData, CreateSignature, VerifySignature). And the RSA class has Encrypt and Decrypt. None of these operations require a certificate.

There's the problem of how one uses the same RSA key later that they did now, with the current answer of RSAParameters / RSA.ImportParameters; and this feature to expand that to PKCS1/PKCS8/EncryptedPkcs8 import.

[KLuuKer]

@bartonjs I need todo a RSA Encrypt with the Public key and Decrypt with the Private key the exact opposite of what the RSA class currently does (yes it sounds dumb but that's how certain folk use it....)

for example when building licensing systems, or making sure you can only read config files instead of writing them (I know a hash would be better but hey I'm not the one making them)

[bartonjs]

Oh! Yeah, that's not something we're very likely to add, partly because not all of the underlying cryptographic libraries we use are capable of doing it; sorry.

[KLuuKer]

No problem I figured as much (you need to support allot), the AsnReader\Writer is already helping allot tough :)

[terrajobst]

Looks good, a few things:

• Replace ReadOnlyMemory<byte> with ReadOnlySpan<byte>. While this complicates the implementation we think having API more consistent is more important (except for the Pkcs8PrivateKeyInfo)
• Rename KdfIterationCount to just IterationCount

[bartonjs]

This was reopened as a 2.2 porting candidate, but that release is feature locked now, so it'll just be 3.0. Since it's already submitted there, no further work required.