[cert] Add option to manually set signature in X509CertificateGenerator.create.

PeculiarVentures / x509

@peculiar/x509 is an easy to use TypeScript/Javascript library based on @peculiar/asn1-schema that makes generating X.509 Certificates and Certificate Requests as well as validating certificate chains easy

https://peculiarventures.github.io/x509/

MIT License

78 stars 10 forks source link

[cert] Add option to manually set signature in X509CertificateGenerator.create. #25

Closed turon closed 1 year ago

turon commented 1 year ago

When creating an x509 certificate manually, field-by-field, for verification purposes, with only the public key, there needs to be an option to set the signature:

        var publicKey = await SubtleCrypto.importKey(
            "raw",
            publicKeyRaw,
            Certificate.Algorithm,
            true, // Extractable?
            ["verify"] 
        )
        var publicKeySpki = await SubtleCrypto.exportKey('spki', publicKey)

       const cert = await x509.X509CertificateGenerator.create ({
            version: 2,
            serialNumber: serialNumber.toString('hex'),
            subject: subject.toString(),
            issuer: issuer.toString(),
            notBefore: new Date(notBefore),
            notAfter: new Date(notAfter),
            signingAlgorithm: Certificate.Algorithm,
            publicKey: publicKeySpki,
            signingKey: publicKey,
            signature: signature,
        })

microshine commented 1 year ago

I don't like the idea of such API usage. For normal usage of the API, we need to sign certificate data by private key. If you have signature, you can use ASN.1 coder and use it to create your custom certificate

Here is updated test slef-signed cert which shows how you can replace signature block for generated certificate. So you can import or generate your own certificate and replace it's any data by yours

TypeScript

import { Crypto } from "@peculiar/webcrypto";
import * as x509 from "@peculiar/x509";
import * as asn1Schema from "@peculiar/asn1-schema";
import { Certificate } from "@peculiar/asn1-x509";

const crypto = new Crypto();
x509.cryptoProvider.set(crypto);

const alg: RsaHashedKeyGenParams = {
  name: "RSASSA-PKCS1-v1_5",
  hash: "SHA-256",
  publicExponent: new Uint8Array([1, 0, 1]),
  modulusLength: 2048,
};
const keys = await crypto.subtle.generateKey(alg, false, ["sign", "verify"]);
assert.ok(keys.publicKey);
assert.ok(keys.privateKey);
const cert = await x509.X509CertificateGenerator.createSelfSigned({
  serialNumber: "01",
  name: "CN=Test, O=Дом",
  notBefore: new Date("2020/01/01"),
  notAfter: new Date("2020/01/02"),
  signingAlgorithm: alg,
  keys: keys,
  extensions: [
    new x509.BasicConstraintsExtension(true, 2, true),
    new x509.ExtendedKeyUsageExtension(["1.2.3.4.5.6.7", "2.3.4.5.6.7.8"], true),
    new x509.KeyUsagesExtension(x509.KeyUsageFlags.keyCertSign | x509.KeyUsageFlags.cRLSign, true),
    new x509.CertificatePolicyExtension([
      "1.2.3.4.5",
      "1.2.3.4.5.6",
      "1.2.3.4.5.6.7",
    ]),
    await x509.SubjectKeyIdentifierExtension.create(keys.publicKey),
  ]
});
const ok = await cert.verify({ date: new Date("2020/01/01 12:00") });
assert.strictEqual(ok, true);

const asnCert = asn1Schema.AsnConvert.parse(cert.rawData, Certificate);
asnCert.signatureValue = new Uint8Array(asnCert.signatureValue.byteLength).buffer;

const raw = asn1Schema.AsnConvert.serialize(asnCert);
console.log(Buffer.from(raw).toString("hex"));

ASN.1 Signature from the created certificate

  SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.1.11 sha256WithRSAEncryption (PKCS #1)
    NULL
  BIT STRING (2048 bit) 000000000000000000000000000000000000000000000000000000000000000000000…

turon commented 1 year ago

I don't like the idea of such API usage. For normal usage of the API, we need to sign certificate data by private key. If you have signature, you can use ASN.1 coder and use it to create your custom certificate

The desired use case is for translation of a TLV encoded / compressed certificate for IoT back into x509 format in order to call cert.verify. In such a case, the user did not create the certificate, nor has the private key. x509.X509CertificateGenerator has good documentation, a very elegant create API (as demonstrated in your example code), and many helpers for the underlying ASN.1 coder, so the hope is that this use case could be supported.

rmhrisk commented 1 year ago

@turon I tend to agree with @microshine however support for common use cases is certainly good. What is the standard that these certificates are used with? How well is it adopted, or is it on target to be adopted broadly?

Do I understand correctly the signature is produced for the subscriber elsewhere? If so a common problem with such systems that verifiers can struggle to verify them do to the variable ways the to be signed representation can be constructed. Maybe the standard talks about how this is handled?

turon commented 1 year ago

@turon I tend to agree with @microshine however support for common use cases is certainly good. What is the standard that these certificates are used with? How will is it adopted, or is it on target to be adopted broadly?

@rmhrisk, the standard is called Matter. The specification is still private to members of the CSA, but the C++ implementation is open and available, including a tool called chip-cert which converts between certificates in PEM, DER, Base64, and this compressed "ChipCert" TLV format. Matter has a very broad set of member companies adopting it, and is on track to be released widely late this year and early 2023.

I'm using @peculiar/x509 to implement an alternative set of tools and utilities for working with these specially encoded x509 certificates in typescript. So far it is working great, far better than alternative packages I've evaluated. During commissioning, a Matter device is provided a Node Operational Certificate signed by the administrator / commissioner. I'd like for the tools to be able to leverage all the x509 infrastructure in the @peculiar packages, both generating and encoding certificates (with private key), but also loading and verifying granted certificates (with only the public key). This later use case requires this PR.

Agree that a developer using this method of constructing a certificate from raw fields can run into problems if the ASN construction isn't exactly right, but in those cases the certificate won't verify, so such problems should be easily to detect and test for. So far the constructed certificates are very close to matching the test vectors with the exception of some name fields being encoded as 'PrintableString' rather than the required 'UTF8String'. Is there a way to specify this encoding preference in x509.X509CertificateGenerator.create()?

microshine commented 1 year ago

Is there a way to specify this encoding preference in x509.X509CertificateGenerator.create()?

Current implementation doesn't allow doing it. It uses ia5String for E and DC, and printableString or utf8String encoding depends on incoming string.

if (type === this.getName("E") || type === this.getName("DC")) {
  attr.value.ia5String = value;
} else {
  // Use Utf8String for non ASCII strings
  if (Name.isASCII(value)) {
    attr.value.printableString = value;
  } else {
    attr.value.utf8String = value;
  }
}

You can do it via ASN.1 schema

import * as asn1Schema from "@peculiar/asn1-schema";
import * as asn1X509 from "@peculiar/asn1-x509";

const asnName = new asn1X509.Name([
  new asn1X509.RelativeDistinguishedName([
    new asn1X509.AttributeTypeAndValue({
      type: "2.5.4.3",
      value: new asn1X509.AttributeValue({
        bmpString: "BMP string",
      }),
    })
  ])
]);

const name = new x509.Name(asn1Schema.AsnConvert.serialize(asnName));
console.log(Convert.ToHex(name.toArrayBuffer()));

ASN.1

SEQUENCE (1 elem)
  SET (1 elem)
    SEQUENCE (2 elem)
      OBJECT IDENTIFIER 2.5.4.3 commonName (X.520 DN component)
      BMPString BMP string

microshine commented 1 year ago

I've created the discussion for name encoding

rmhrisk commented 1 year ago

I am aware of Matter and have reviewed some related documentation but have not had a chance to review the protocol documentation.

Do the Matter specs explicitly state what restricted encoding set to use to ensure this TLV->X509 works reliably?

Though your right these will be signature failures and would certainly require debugging I suspect the nature of failures will be seen in the field in random cases. It seems a specification would need to be quite explicit to ensure interop.

Do I understand correctly your library is intended to implement both subscriber and verifier?

Will this library be public at some point?

I don’t like the idea of exposing more esoteric options in the x509 interface, however if this is necessary to deliver a complete library and we can’t think of a better way to enable it cleanly I guess I’m OK with it.

I imagine these two items are just the top of the iceberg, are your far enough along to know with a complete change list?

turon commented 1 year ago

Do the Matter specs explicitly state what restricted encoding set to use to ensure this TLV->X509 works reliably?

Though your right these will be signature failures and would certainly require debugging I suspect the nature of failures will be seen in the field in random cases. It seems a specification would need to be quite explicit to ensure interop.

Yes, the restricted encoding rules are precisely specified, and your concerns regarding interop are correct. In fact, part of the motivation for a typescript implementation would be to confirm such interop.

Do I understand correctly your library is intended to implement both subscriber and verifier?

Will this library be public at some point?

Yes, the library will both generate and consume certificates of various sorts (Root, Node Operational, Device Attestation, ...). The intention is for this typescript library to be public; it is currently in prototyping phase however.

I don’t like the idea of exposing more esoteric options in the x509 interface, however if this is necessary to deliver a complete library and we can’t think of a better way to enable it cleanly I guess I’m OK with it.

I imagine these two items are just the top of the iceberg, are your far enough along to know with a complete change list?

No, I haven't identified a complete change list yet to enable complete support for Matter certificates. Once the TLV encoded certificates can be loaded as x509 and successfully verified, the hope is that the current set of x509 functionality will be sufficient. Understood that the API should remain minimal and targeted. Thanks for the openness to adding the ability to load a certificate as a verifier in a non-standard format (not PEM or DER). I think it is a good addition for a flexible and powerful x509 library such as this one.

turon commented 1 year ago

@turon Let's support this feature.

But I've got a suggestion about your PR. Your code adds one more required option into X509CertificateCreateParams. I guess usage of mixed type would be better.

Great! I'll integrate your suggestions.

turon commented 1 year ago

The result of const asnName = new asn1X509.Name doesn't seem to pass the check for if (data instanceof asn1X509.Name).

Is there any documentation, example, or test reference for how to use Name.fromJSON?

From the code I've deduced that the format is:

        const issuerJsonName = [{
            "1.3.6.1.4.1.37244.1.3": [icacIdHex]
        }]

This Json API is bit more intuitive and natural for single name encoding versus fromString API.

Would it be reasonable to expand the jsonName API to allow explicit encoding options via object?

        const issuerJsonName = [{
            "1.3.6.1.4.1.37244.1.3": [{utf8String: icacIdHex}]
        }]

microshine commented 1 year ago

Is there any documentation, example, or test reference for how to use Name.fromJSON?

No, there isn't.

Example

const name = new x509.Name([
  {
    CN: ["Name"],
  },
  {
    O: ["Organization"],
  }
]);
assert.strictEqual(name.toString(), "CN=Name, O=Organization");
console.log(asn1Schema.AsnConvert.toString(name.toArrayBuffer()));

ASN

SEQUENCE :
  SET :
    SEQUENCE :
      OBJECT IDENTIFIER : 2.5.4.3
      PrintableString : 'Name'
  SET :
    SEQUENCE :
      OBJECT IDENTIFIER : 2.5.4.10
      PrintableString : 'Organization'

Would it be reasonable to expand the jsonName API to allow explicit encoding options via object?

I have the same suggestion in https://github.com/PeculiarVentures/x509/discussions/27#discussioncomment-3115103

But this JSON schema (and current) doesn't allow creating the name like CN=First+O=Org+CN=Second

const name = new x509.Name("CN=First+O=Org+CN=Second");
console.log(name.toJSON());
// [ { CN: [ 'First', 'Second' ], O: [ 'Org' ] } ]

const name2 = new x509.Name(name.toJSON());
console.log(name2.toString());
// CN=First+CN=Second+O=Org

I suppose we could update toJSON/fromJSON methods for the Name class and publish minor version. I don't think anybody uses Name JSON API.

turon commented 1 year ago

I have the same suggestion in #27 (comment)

Great! Well that would be a separate PR, but glad to know the general direction is an approved one.

I suppose we could update toJSON/fromJSON methods for the Name class and publish minor version. I don't think anybody uses Name JSON API.

The Name JSON API has proven to be very good for the purpose of realizing this use case.

So as an update, I am able to generate a certificate which outputs equivalent PEM and DER that match the expected test vectors using this path. The signature doesn't verify however. I'm not sure why that would be. If I run cert.toString("pem"), it outputs the following, which seems to verify on certificate viewer sites. Does signing with the private key perform some side effects on the internal object state that are required for verify?

microshine commented 1 year ago

The signature doesn't verify however. I'm not sure why that would be

Is it possible to get the original certificate, or it's tbsCertificate on signing? It would be very helpful for the problem understanding.

To get a valid certificate, you need to reproduce the same tbsCertificate (see RFC 5280) It's possible that original signatureAlgorithms is different.

SEQUENCE (1 elem)
  OBJECT IDENTIFIER 1.2.840.10045.4.3.2 ecdsaWithSHA256 (ANSI X9.62 ECDSA algorithm with SHA256)
+ NULL

turon commented 1 year ago

Yes, here are the complete set of test vectors:

[Root CA]

[Intermediate CA]

[Operational Certificate] (@peculiar/x509 .toString('pem') is exact match here, but verify fails)

I'm assuming I may need to pass the CA certificate chain to .verify()?

signatureAlgorithm is indeed ecdsa-with-SHA256 NIST P-256 curve.

I was actually slightly confused by this line:

    const signingAlgorithm = { ...params.signingAlgorithm, ...params.signingKey.algorithm };

Which parameters exactly are expected the get picked up by the ...spread operator? Wouldn't the two always be equal? In my case, both are expected to equal the object below:

    static Algorithm = {
        name: "ECDSA",
        namedCurve: "P-256",
        hash: "SHA-256",
    }

microshine commented 1 year ago

I was actually slightly confused by this line:

const signingAlgorithm = { ...params.signingAlgorithm, ...params.signingKey.algorithm };

The problem is in WebCrypto key algorithms.

RSA key algorithm contains hash algorithm. That means that you can't change the hash algorithm on signing. ECDSA key algorithm doesn't have hash algorithm, this is why we take that value from the params.signingAlgorithm

const rsaSigningAlg = {
  ...{ // params.signingAlgorithm
    name: "RSASSA-PKCS1-v1_5",
  },
  ...{ // params.signingKey.algorithm
    name: "RSASSA-PKCS1-v1_5",
    hash: {
      name: "SHA-256",
    },
    publicExponent: Uint8Array([1,0,1]),
    publicExponent: 2048
  },
}
/*
// combined RSA algorithm
{
  name: "RSASSA-PKCS1-v1_5",
  hash: {
    name: "SHA-256",
  },
  publicExponent: Uint8Array([1,0,1]),
  publicExponent: 2048
}
*/
const ecdsaSigningAlg = {
  ...{ // params.signingAlgorithm
    name: "ECDSA",
    hash: {
      name: "SHA-256"
    }
  },
  ...{ // params.signingKey.algorithm
    name: "ECDSA",
    namedCurve: "P-256"
  },
}
/*
// combined ECDSA algorithm
{
  name: "ECDSA",
  hash: {
    name: "SHA-256",
  },
  namedCurve: "P-256"
}
*/

My test is passed

const ca = new x509.X509Certificate("-----BEGIN CERTIFICATE-----\nMIIBnTCCAUOgAwIBAgIILbREhVZBrt8wCgYIKoZIzj0EAwIwIjEgMB4GCisGAQQB\ngqJ8AQQMEENBQ0FDQUNBMDAwMDAwMDEwHhcNMjAxMDE1MTQyMzQzWhcNNDAxMDE1\nMTQyMzQyWjAiMSAwHgYKKwYBBAGConwBAwwQQ0FDQUNBQ0EwMDAwMDAwMzBZMBMG\nByqGSM49AgEGCCqGSM49AwEHA0IABMXQhhu4+QxAXBIxTkxevuqTn3J3S8wzI54v\nWfb0avjcfUaCoOPMxkbm3ynqhr9WKucgqJgzfTg/MsCgnkFgGeqjYzBhMA8GA1Ud\nEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMB0GA1UdDgQWBBRTUtcFnpwVpQiQ\naGKGSAGinx9B0zAfBgNVHSMEGDAWgBQTr4GrNzdLLtKpZJsSt6OkKH4VHTAKBggq\nhkjOPQQDAgNIADBFAiEAhBoG1Dten+zSToexJE61HGos8g2bXmugfxHmAC9+DKMC\nIE4ypgLDYJ0AktNIvb0ZihFGRr1BzxA3g2Qa4l4/I/0m\n-----END CERTIFICATE-----");
const cert = new x509.X509Certificate("-----BEGIN CERTIFICATE-----\nMIIB4DCCAYagAwIBAgIIPvz/FwK5oXowCgYIKoZIzj0EAwIwIjEgMB4GCisGAQQB\ngqJ8AQMMEENBQ0FDQUNBMDAwMDAwMDMwHhcNMjAxMDE1MTQyMzQzWhcNNDAxMDE1\nMTQyMzQyWjBEMSAwHgYKKwYBBAGConwBAQwQREVERURFREUwMDAxMDAwMTEgMB4G\nCisGAQQBgqJ8AQUMEEZBQjAwMDAwMDAwMDAwMUQwWTATBgcqhkjOPQIBBggqhkjO\nPQMBBwNCAASaKiFvs53WtvohG4NciePmr7ZsFPdYMZVPn/T3o/ARLIoNjq8pxlMp\nTUju4HCKAyzKOTk8OntG8YGuoHj+rYODo4GDMIGAMAwGA1UdEwEB/wQCMAAwDgYD\nVR0PAQH/BAQDAgeAMCAGA1UdJQEB/wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATAd\nBgNVHQ4EFgQUn1Wia35DA+YIg+kTv5T0+14qYWEwHwYDVR0jBBgwFoAUU1LXBZ6c\nFaUIkGhihkgBop8fQdMwCgYIKoZIzj0EAwIDSAAwRQIgeVXCAmMLS6TVkSUmMi/f\nKPie3+WvnA5XK9ihSqq7TRICIQC4PKF8ewX7Fkt315xSlhMxa8/ReJXksqTyQEuY\nFzJxWQ==\n-----END CERTIFICATE-----");

const ok = await cert.verify({
  publicKey: ca,
  signatureOnly: true,
}, crypto);
assert.strictEqual(ok, true);

turon commented 1 year ago

@microshine, verify works now that the API is called correctly. Thanks for all the excellent code samples!

turon commented 1 year ago

@microshine changes posted according to your feedback. Please take a look.

microshine commented 1 year ago

Please fix test errors and lint warnings

turon commented 1 year ago

Try to use numbers for date creation instead of strings

notBefore: new Date(1577865600000), // UTCTime 2020-01-01 08:00:00 UTC
notAfter: new Date(2209104000000), // UTCTime 2040-01-02 08:00:00 UTC

JavaScript Date just seems broken in so many ways. Docs say init string without time will be UTC. .setSeconds also has strange timezone bugs. I'll try your suggestion if I can't get Date.UTC to work.

UPDATE: fixed issues with Date in test.

microshine commented 1 year ago

@peculiar/x509@1.8.0 has been published