search

bcgit / bc-java

Bouncy Castle Java Distribution (Mirror)
https://www.bouncycastle.org/java.html
MIT License
2.29k stars 1.13k forks source link

BouncyCastle Ed25519 ASN.1 encoding seems to be incompatible with OpenSSL #1029

Closed hojothum closed 2 years ago

hojothum commented 3 years ago

When generating ed25519 key-pairs in Java, using BC, and then writing them out to PEM files for use with OpenSSL, we have encountered error messages when OpenSSL tries to read the key files.

Code for generating the keys and exporting to PEM file:

        Security.addProvider(new BouncyCastleProvider());

        KeyPairGenerator kpg = KeyPairGenerator.getInstance("Ed25519", "BC");
        EdDSAParameterSpec edDSAParameterSpec = new EdDSAParameterSpec(EdDSAParameterSpec.Ed25519);
        kpg.initialize(edDSAParameterSpec, SecureRandom.getInstanceStrong());
        KeyPair keyPair = kpg.generateKeyPair();

        try (BufferedWriter writer = Files.newBufferedWriter(Path.of("/tmp/","bc_ed25519_sk.pem"), StandardCharsets.US_ASCII);
             JcaPEMWriter pemWriter = new JcaPEMWriter(writer)) {
            pemWriter.writeObject(keyPair.getPrivate());
            pemWriter.flush();
        }

OpenSSL (1.1.1l & 3.0.0) attempt to parse PEM from Java:

v1.1.1l

/usr/local/opt/openssl@1.1/bin/openssl pkey -in /tmp/bc_ed25519_sk.pem -text
unable to load key
4445879808:error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag:crypto/asn1/tasn_dec.c:1149:
4445879808:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:572:Field=attributes, Type=PKCS8_PRIV_KEY_INFO
4445879808:error:0907B00D:PEM routines:PEM_read_bio_PrivateKey:ASN1 lib:crypto/pem/pem_pkey.c:88:

/usr/local/opt/openssl@1.1/bin/openssl pkey -in /tmp/bc_ed25519_sk.pem -pubout -out /tmp/bc_ed25519_pk.pem
unable to load key
4331810304:error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag:crypto/asn1/tasn_dec.c:1149:
4331810304:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:572:Field=attributes, Type=PKCS8_PRIV_KEY_INFO
4331810304:error:0907B00D:PEM routines:PEM_read_bio_PrivateKey:ASN1 lib:crypto/pem/pem_pkey.c:88:

/usr/local/opt/openssl@1.1/bin/openssl asn1parse -in /tmp/bc_ed25519_sk.pem
    0:d=0  hl=2 l=  81 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :01
    5:d=1  hl=2 l=   5 cons: SEQUENCE          
    7:d=2  hl=2 l=   3 prim: OBJECT            :ED25519
   12:d=1  hl=2 l=  34 prim: OCTET STRING      [HEX DUMP]:0420B500BAA5C616058C5AA6186FA43DCEF6A4516BA21A8DCC7805CAEC6B2706739F
   48:d=1  hl=2 l=  33 prim: cont [ 1 ]

Local build of v3.0.0 tag:

env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl version
OpenSSL 3.0.0 7 sep 2021 (Library: OpenSSL 3.0.0 7 sep 2021)

env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl pkey -in /tmp/bc_ed25519_sk.pem -text
Could not read key from /tmp/bc_ed25519_sk.pem
008E271901000000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:crypto/store/store_result.c:151:

env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl pkey -in /tmp/bc_ed25519_sk.pem -pubout -out /tmp/bc_ed25519_pk.pem
Could not read key from /tmp/bc_ed25519_sk.pem
008EE80801000000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:crypto/store/store_result.c:151:

env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl asn1parse -in /tmp/bc_ed25519_sk.pem
    0:d=0  hl=2 l=  81 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :01
    5:d=1  hl=2 l=   5 cons: SEQUENCE          
    7:d=2  hl=2 l=   3 prim: OBJECT            :ED25519
   12:d=1  hl=2 l=  34 prim: OCTET STRING      [HEX DUMP]:0420B500BAA5C616058C5AA6186FA43DCEF6A4516BA21A8DCC7805CAEC6B2706739F
   48:d=1  hl=2 l=  33 prim: cont [ 1 ]

There seems to be a clear difference in the ASN.1 encoding when comparing the PEM output of OpenSSL to that of Java. Maybe that implies that the ASN.1 encoding produced by BouncyCastle is incompatible with what OpenSSL is expecting for ed25519 keys?

/usr/local/opt/openssl@1.1/bin/openssl asn1parse -in /tmp/ossl_ed25519_sk.pem 
    0:d=0  hl=2 l=  46 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :00
    5:d=1  hl=2 l=   5 cons: SEQUENCE          
    7:d=2  hl=2 l=   3 prim: OBJECT            :ED25519
   12:d=1  hl=2 l=  34 prim: OCTET STRING      [HEX DUMP]:0420FE467C74FFE7295C6CC5F7DA411207DA02BCB9C03D475B84C97708850584B35A

/usr/local/opt/openssl@1.1/bin/openssl asn1parse -in /tmp/bc_ed25519_sk.pem
    0:d=0  hl=2 l=  81 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :01
    5:d=1  hl=2 l=   5 cons: SEQUENCE          
    7:d=2  hl=2 l=   3 prim: OBJECT            :ED25519
   12:d=1  hl=2 l=  34 prim: OCTET STRING      [HEX DUMP]:0420B500BAA5C616058C5AA6186FA43DCEF6A4516BA21A8DCC7805CAEC6B2706739F
   48:d=1  hl=2 l=  33 prim: cont [ 1 ]

Verifying OpenSSL can handle ed25519 internally:

/usr/local/opt/openssl@1.1/bin/openssl version
OpenSSL 1.1.1l  24 Aug 2021

/usr/local/opt/openssl@1.1/bin/openssl genpkey -algorithm Ed25519 -out /tmp/ossl_ed25519_sk.pem
/usr/local/opt/openssl@1.1/bin/openssl pkey -in /tmp/ossl_ed25519_sk.pem -text
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIP5GfHT/5ylcbMX32kESB9oCvLnAPUdbhMl3CIUFhLNa
-----END PRIVATE KEY-----
ED25519 Private-Key:
priv:
    fe:46:7c:74:ff:e7:29:5c:6c:c5:f7:da:41:12:07:
    da:02:bc:b9:c0:3d:47:5b:84:c9:77:08:85:05:84:
    b3:5a
pub:
    7a:93:c7:c8:a1:16:ff:95:d2:db:4f:06:97:c4:84:
    f2:6d:b7:60:a0:e8:f5:3f:c6:55:4f:17:54:e2:f4:
    7f:a4

/usr/local/opt/openssl@1.1/bin/openssl pkey -in /tmp/ossl_ed25519_sk.pem -pubout -out /tmp/ossl_ed25519_pk.pem 

/usr/local/opt/openssl@1.1/bin/openssl asn1parse -in /tmp/ossl_ed25519_sk.pem 
    0:d=0  hl=2 l=  46 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :00
    5:d=1  hl=2 l=   5 cons: SEQUENCE          
    7:d=2  hl=2 l=   3 prim: OBJECT            :ED25519
   12:d=1  hl=2 l=  34 prim: OCTET STRING      [HEX DUMP]:0420FE467C74FFE7295C6CC5F7DA411207DA02BCB9C03D475B84C97708850584B35A

/usr/local/opt/openssl@1.1/bin/openssl asn1parse -in /tmp/ossl_ed25519_pk.pem
    0:d=0  hl=2 l=  42 cons: SEQUENCE          
    2:d=1  hl=2 l=   5 cons: SEQUENCE          
    4:d=2  hl=2 l=   3 prim: OBJECT            :ED25519
    9:d=1  hl=2 l=  33 prim: BIT STRING

The same strategy can be used to export PEM files from Java/BC for ECDSA keys and it works as expected:

        Security.addProvider(new BouncyCastleProvider());

        KeyPairGenerator kpg = KeyPairGenerator.getInstance("ECDSA", "BC");
        ECGenParameterSpec ecGenParameterSpec = new ECGenParameterSpec("secp256k1");
        kpg.initialize(ecGenParameterSpec, SecureRandom.getInstanceStrong());
        KeyPair keyPair = kpg.generateKeyPair();

        try (BufferedWriter writer = Files.newBufferedWriter(Path.of("/tmp/","bc_secp256k1_sk.pem"), StandardCharsets.US_ASCII);
             JcaPEMWriter pemWriter = new JcaPEMWriter(writer)) {
            pemWriter.writeObject(keyPair.getPrivate());
            pemWriter.flush();
        }
env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl pkey -in /tmp/bc_secp256k1_sk.pem -text
-----BEGIN PRIVATE KEY-----
MIGEAgEAMBAGByqGSM49AgEGBSuBBAAKBG0wawIBAQQgO9J0+gWGO9TwJ4G06zYV
dMyMt9XKlfe4kCaJo2+VcoShRANCAAT6EB2ZsA6PI0eO5tigfPFDWCCURgYQTj0m
I0T7pcfvCDS3MFKi8xa8iXGaxXP52HJykHtrOZzWkXBDJ7A6baE1
-----END PRIVATE KEY-----
Private-Key: (256 bit)
priv:
    3b:d2:74:fa:05:86:3b:d4:f0:27:81:b4:eb:36:15:
    74:cc:8c:b7:d5:ca:95:f7:b8:90:26:89:a3:6f:95:
    72:84
pub:
    04:fa:10:1d:99:b0:0e:8f:23:47:8e:e6:d8:a0:7c:
    f1:43:58:20:94:46:06:10:4e:3d:26:23:44:fb:a5:
    c7:ef:08:34:b7:30:52:a2:f3:16:bc:89:71:9a:c5:
    73:f9:d8:72:72:90:7b:6b:39:9c:d6:91:70:43:27:
    b0:3a:6d:a1:35
ASN1 OID: secp256k1

env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl asn1parse -in /tmp/bc_secp256k1_sk.pem 
    0:d=0  hl=2 l= 116 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :01
    5:d=1  hl=2 l=  32 prim: OCTET STRING      [HEX DUMP]:3BD274FA05863BD4F02781B4EB361574CC8CB7D5CA95F7B8902689A36F957284
   39:d=1  hl=2 l=   7 cons: cont [ 0 ]        
   41:d=2  hl=2 l=   5 prim: OBJECT            :secp256k1
   48:d=1  hl=2 l=  68 cons: cont [ 1 ]    

env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl pkey -in /tmp/bc_secp256k1_sk.pem -pubout -out /tmp/bc_secp256k1_pk.pem

env LD_LIBRARY_PATH=(pwd)/build/lib/ ./build/bin/openssl asn1parse -in /tmp/bc_secp256k1_pk.pem
    0:d=0  hl=2 l=  86 cons: SEQUENCE          
    2:d=1  hl=2 l=  16 cons: SEQUENCE          
    4:d=2  hl=2 l=   7 prim: OBJECT            :id-ecPublicKey
   13:d=2  hl=2 l=   5 prim: OBJECT            :secp256k1
   20:d=1  hl=2 l=  66 prim: BIT STRING
hojothum commented 3 years ago

Apologies, meant to include the BC version. It's pulled from Maven repos:

        <dependency>
            <groupId>org.bouncycastle</groupId>
            <artifactId>bcprov-jdk15on</artifactId>
            <version>1.69</version>
        </dependency>
        <dependency>
            <groupId>org.bouncycastle</groupId>
            <artifactId>bcpkix-jdk15on</artifactId>
            <version>1.69</version>
        </dependency>
tonywasher commented 3 years ago

BouncyCastle is outputting the Private key in the format described by RFC 5958, as can be seen by the initial integer (version) field being 1 rather than 0.

RFC 5958 obsoletes RFC 5208 which describes the version 0 format, which it appears is the only one supported by openSSL.

You can request BouncyCastle to output the old v0 format that is acceptable to openSSL by setting the system property org.bouncycastle.pkcs8.v1_info_only. This results in a pem file that is acceptable to openSSL. Just add -Dorg.bouncycastle.pkcs8.v1_info_only=true to the VM options of your process.

hojothum commented 2 years ago

Perfect. Thank you for educating me on what's going on underneath and for the solution via VM param.

Running the test application with java -Dorg.bouncycastle.pkcs8.v1_info_only=true ...

This now works:

/usr/local/opt/openssl@1.1/bin/openssl pkey -in /tmp/bc_ed25519_sk.pem -text
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEICxled/CNt2EpNiK2vCbFSbpcIFi7fr1pKmq7WC2LtOJ
-----END PRIVATE KEY-----
ED25519 Private-Key:
priv:
    2c:65:79:df:c2:36:dd:84:a4:d8:8a:da:f0:9b:15:
    26:e9:70:81:62:ed:fa:f5:a4:a9:aa:ed:60:b6:2e:
    d3:89
pub:
    7d:b6:d4:54:f8:7d:37:bc:4a:e7:82:d7:42:49:9d:
    49:fa:28:79:04:73:5d:0f:b6:0f:2d:a6:e1:f5:6f:
    ed:5f
/usr/local/opt/openssl@1.1/bin/openssl asn1parse -in /tmp/bc_ed25519_sk.pem
    0:d=0  hl=2 l=  46 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :00
    5:d=1  hl=2 l=   5 cons: SEQUENCE          
    7:d=2  hl=2 l=   3 prim: OBJECT            :ED25519
   12:d=1  hl=2 l=  34 prim: OCTET STRING      [HEX DUMP]:04202C6579DFC236DD84A4D88ADAF09B1526E9708162EDFAF5A4A9AAED60B62ED389