Fail to pass signature verification

[bafu]

Created a CAI-injected photo, and failed to pass signature verification.

Steps to Reproduce

1. Create a keypair by following README:

   $ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 7 $ openssl pkcs12 -export -out certificate.p12 -inkey key.pem -in cert.pem $ openssl rsa -in key.pem -pubout -out pubkey.pem

2. Create a CAI-injected testing photo

   Download and extract cicada.zip, and replace my key.pem, pubkey.pem, certificate.p12, cert.pem to yours.

   $ cd cicada $ ./run.sh cicada.jpg

   You will get a CAI-injected photo cicada-cai.jpg.

3. Get Claim indented JSON and Signature HEX string from CAI JSON parser

   caifile.py

   $ python3 caifile.py cicada.jpg

4. Convert Claim indented JSON to the Claim JSON for Signature

   You can not remove spaces directly because the value of some fields contains space(s). You need to remove spaces like json_to_bytes.

5. Convert the Signature HEX string to Signature binary

   Copy and paste the Signature HEX string to sighex as the input file and run hex2bin.py to generate sigbin as the output file:

   $ python3 hex2bin.py sighex sigbin

6. Verify the Claim and its Signature

   $ openssl pkcs7 -inform der -in sigbin -out sigbin.pkcs7 $ openssl pkcs7 -print_certs -in sigbin.pkcs7 -out sigbin.cert $ openssl smime -verify -binary -inform der -in sigbin.der -content claim.json -certfile sigbin.cert -noverify

   You will get a verification failure.

   Followed the Signature tutorial and also failed to verify:

   $ python3 digital_signature.py -v pubkey.pem claim.json sigbin Verifying Signature Failure

Environment

• starling-cai: commit 42572cc (fix-fail-to-verify-signature branch)
• codestream-parser: commit f0f97c4 (feature-caifile-parser branch)

[ethanwu155]

The above signature verification is for the cms signature implementation that is not the recommended digital signature implementation. I have added the following endesive CADES-B signature implementation in the utils folder. It includes endesive_sign.py script for signing and verifying. README also available with steps Adobe took to verify signature.

Summary:

General p12 and cert.pem

1. Generate Public and Private Key with the following: openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 7 openssl pkcs12 -export -out <filename>.p12 -inkey key.pem -in cert.pem

   Output will be the following:

   <filename>.p12

2. Generate certificate for verification with the following: openssl pkcs12 -in <filename>.p12 -out <filename>crt.pem -clcerts -nokeys

   Output will be the following:

   <filename>.crt.pem

3. Generate Signature with:

   python endesive-sign.py -s <p12> <claim JSON> <name of signature file.der>

   4. Verify Signature with:

      python endesive-sign.py -v <crt.pem> <claim JSON> <signature file .der>

Sample Usage:

# claim json : starling.claim.json

# Generate p12 
# Will output file certificate.p12 (will be used to generate signature). set pass to 1234
$ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 7
$ openssl pkcs12 -export -out certificate.p12 -inkey key.pem -in cert.pem

# Generate crt.pem
# Will output file certificate.crt.pem (will be used to verify signed) 
$ openssl pkcs12 -in certificate.p12 -out certificate.crt.pem -clcerts -nokeys

# Usage:
# Generating Signature:
$ python endesive-sign.py -s certificate.p12 starling.claim.json starling.der
Generating Signature

# Verifying Signature:
$ python endesive-sign.py -v certificate.crt.pem starling.claim.json starling.der
Verifying Signature
signature ok? True
hash ok? True
cert ok? True

Verifying using Adobe's Methodology

Requirements

• Extracted CMS signature to starling.der
• Extracted claim JSON to starling.claim.json (exact byte sequence inserted into image)

# Convert signature from DER to PEM encoding:
$ openssl pkcs7 -inform der -in starling.der -out starling.der.pkcs7

# Extract X.509 certificates from signature:
$ openssl pkcs7 -print_certs -in starling.der.pkcs7 -out starling.der.cert

# Verify CMS signature against detached data (claim):
$ openssl smime -verify -binary -inform der -in starling.der -content starling.claim.json -certfile starling.der.cert -noverify
{
    "assertions": [
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.location.precise?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMoBUAVbTEB7K",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.sensors?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMvY4QFEN3973",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.device?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMwEoBojZcUrZ",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMo3SG72sZg13"
    ],
    "asset_hashes": [
        {
            "start": "0x0000000000000000",
            "length": "0x0000000000009959",
            "name": "JFIF SOI-APP0",
            "url": "",
            "value": "EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="
        },
        {
            "start": "0x0000000000009959",
            "length": "0x000000000000027d",
            "name": "JFIF APP1/XMP",
            "url": "",
            "value": "EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="
        },
        {
            "start": "0x000000000000a90c",
            "length": "0x00000000000215e6",
            "name": "JFIF DQT-EOI",
            "url": "",
            "value": "EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="
        }
    ],
    "recorder": "Starling Capture",
    "signature": "self#jumbf=cai/cb.starling_1/cai.signature"
Verification successful

[ethanwu155]

Added tutorial portion of signature-verification. Also was able to set passphrase for p12 to ''.

See Below:

# Export existing pkcs12 to pem file
$ openssl pkcs12 -in certificate.p12 -nodes -out temp.pem
Enter Import Password:
MAC verified OK

# convert pem back to p12 w/ no password
$ openssl pkcs12 -export -in temp.pem  -out unprotected.p12
Enter Export Password:
Verifying - Enter Export Password:

# remove temp certfiicate
$ rm temp.pem

# generate crt.pem
$ openssl pkcs12 -in unprotected.p12 -out unprotected.crt.pem -clcerts -nokeys
Enter Import Password:
MAC verified OK

# sign claim
$ python endesive-sign.py -s unprotected.p12 starling.claim.json unprotected.der
Generating Signature

# verify signature
$ python endesive-sign.py -v unprotected.crt.pem starling.claim.json unprotected.der 
Verifying Signature
signature ok? True
hash ok? True
cert ok? True

Adobe Verification works the same way:

$ openssl pkcs7 -inform der -in unprotected.der -out unprotected.der.pkcs7

$ openssl pkcs7 -print_certs -in unprotected.der.pkcs7 -out unprotected.der.cert

$ openssl smime -verify -binary -inform der -in unprotected.der -content starling.claim.json -certfile unprotected.der.cert -noverify
{
    "assertions": [
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.location.precise?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMoBUAVbTEB7K",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.sensors?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMvY4QFEN3973",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.device?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMwEoBojZcUrZ",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMo3SG72sZg13"
    ],
    "asset_hashes": [
        {
            "start": "0x0000000000000000",
            "length": "0x0000000000009959",
            "name": "JFIF SOI-APP0",
            "url": "",
            "value": "EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="
        },
        {
            "start": "0x0000000000009959",
            "length": "0x000000000000027d",
            "name": "JFIF APP1/XMP",
            "url": "",
            "value": "EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="
        },
        {
            "start": "0x000000000000a90c",
            "length": "0x00000000000215e6",
            "name": "JFIF DQT-EOI",
            "url": "",
            "value": "EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="
        }
    ],
    "recorder": "Starling Capture",
    "signature": "self#jumbf=cai/cb.starling_1/cai.signature"
Verification successful

CAI code has been modified to take no passphrase:

elif type_sig=='endesive':
                # load_key_and_certificates second parameter is password to decrypt the data. Can be set to None of PKCS12 is not encrypted
                # https://cryptography.io/en/latest/hazmat/primitives/asymmetric/serialization.html
                key = pkcs12.load_key_and_certificates(f.read(), b'', backends.default_backend())

[bafu]

@ethanwu155 I followed the steps but the hash check is False.

My steps and testing data (cicada.zip) are below. Can you help check it? Thanks!

$ endesive-sign.py -v certificate.crt.pem claim.json claim-signature.der
Verifying Signature
signature ok? True
hash ok? False
cert ok? True

[ethanwu155]

The reason hash is wrong is because extracted claim.json is incorrect. Currently extracted claim.json from testing data is (includes label and content):

{"label":"cai.claim","content":{"assertions":["self#jumbf=cai/cb.starling_1/cai.assertions/cai.location.broad?hl=mEiAiWeYKpcrzcEZUwo1l1J09GPElaDfnEarwHnJvlAh/oA","self#jumbf=cai/cb.starling_1/cai.assertions/cai.rights?hl=mEiDz9sNp/tzDru0T9PjM6InZBzjMLlg0XX6kYTbs8DtdJw","self#jumbf=cai/cb.starling_1/cai.assertions/cai.claim.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/cai.acquisition.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/adobe.asset.info?hl=mEiCm/aYUgYWjwC0emm90PWK2qxpbXxqC6/bMJMkKevY1LA","self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=mEiBX5zocwF840JwzsiC+LHPubE9MTn93t8utThK/kguMsg"],"asset_hashes":[{"length":"0x0000000000009959","name":"JFIF SOI-APP0","start":"0x0000000000000000","url":"","value":"EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="},{"length":"0x000000000000027d","name":"JFIF APP1/XMP","start":"0x0000000000009959","url":"","value":"EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="},{"length":"0x00000000000215e6","name":"JFIF DQT-EOI","start":"0x000000000000a90c","url":"","value":"EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="}],"recorder":"Starling Capture using Numbers Protocol","signature":"self#jumbf=cai/cb.starling_1/cai.signature"}}

Signed claim should be:

{"assertions":["self#jumbf=cai/cb.starling_1/cai.assertions/cai.location.broad?hl=mEiAiWeYKpcrzcEZUwo1l1J09GPElaDfnEarwHnJvlAh/oA","self#jumbf=cai/cb.starling_1/cai.assertions/cai.rights?hl=mEiDz9sNp/tzDru0T9PjM6InZBzjMLlg0XX6kYTbs8DtdJw","self#jumbf=cai/cb.starling_1/cai.assertions/cai.claim.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/cai.acquisition.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/adobe.asset.info?hl=mEiCm/aYUgYWjwC0emm90PWK2qxpbXxqC6/bMJMkKevY1LA","self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=mEiBX5zocwF840JwzsiC+LHPubE9MTn93t8utThK/kguMsg"],"asset_hashes":[{"length":"0x0000000000009959","name":"JFIF SOI-APP0","start":"0x0000000000000000","url":"","value":"EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="},{"length":"0x000000000000027d","name":"JFIF APP1/XMP","start":"0x0000000000009959","url":"","value":"EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="},{"length":"0x00000000000215e6","name":"JFIF DQT-EOI","start":"0x000000000000a90c","url":"","value":"EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="}],"recorder":"Starling Capture using Numbers Protocol","signature":"self#jumbf=cai/cb.starling_1/cai.signature"}

If using the correct claim contents:

$ python endesive-sign.py -v test/certificate.crt.pem test/claim.json test/claim-signature.der 
Verifying Signature
signature ok? True
hash ok? True
cert ok? True

[bafu]

When creating claim.json manually, you might need to remove the '0x0a' at the end of the file.

Check if there is 0x0a at the end of the file:

$ xxd claim.json
00000000: 7b22 6173 7365 7274 696f 6e73 223a 5b22  {"assertions":["
...
00000510: 7265 227d 0a                             re"}.

Remove 0x0a by Vim as example:

$ vim -b claim.json

# In Vim
:set noeol
:wq

Double check if there is 0x0a:

$ xxd claim.json
00000000: 7b22 6173 7365 7274 696f 6e73 223a 5b22  {"assertions":["
...
00000510: 7265 227d                                re"}

[bafu]

Closed by PR #18