Verification of Starling Capture signatures

benhylau commented 2 years ago

Task Summary

📅 Due date: N/A
🎯 Success criteria: Verify cryptographic signatures that come in through Starling Capture app.

The integrity-backend currently does not verify signatures from the HTC Exodus phones. We simply rely on the JWT authorization, and proceed to trust that the photo came from the provisioned phone, but in fact, the phone provides two signatures signing the image hash and metadata bundle. We should be verifying both signatures as part of our secure workflow.

The POST message contains meta and signature fields that need to be verified.

numbers-AndroidOpenSSL

This is the software signature using OpenSSL. This is what Numbers uses to verify the signature, we can take from this implementation: https://github.com/numbersprotocol/starling-capture/blob/master/util/verification/verification/verification.py

numbers-Zion

This is the hardware secure enclave signature. The output of the verification is the Ethereum wallet address associated with the phone (we set this up when provisioning the device).

We should take the meta and signature fields, and generate the Ethereum address. The address output should be recorded as content-metadata, and serves as a hardware identifier of the recording device.

starling-capture-test-asset.zip

To Do

[x] Scaffold an interface that allows verification of different signaturing algos (starting with the two listed here)
[x] numbers-AndroidOpenSSL verification
[x] numbers-Zion verification + include output in content metadata

benhylau commented 2 years ago

On how these "point of capture" sigs relate to "c2pa / authsign" sigs

The flow goes like this:

Device sends us stuff using their supported scheme (e.g. openssl, ethereum, issue-66-custom-scheme, cole-homemade-crypto-sign)
integrity-backend looks at the source, and decides "oh I know how to verify this", does the verification
If signature checks out there, then integrity-backend uses the Starling key to give it a stamp of approval, attesting that "yea I checked it out, this is real", and that "notary" key could be C2PA key, authsign on a starlinglab.org domain, etc.

One analogy could be:

I take a letter to the notary, the lawyer there asks "show me your photo ID" and I can give any supported ID (e.g. drivers, passport)
lawyer at the notary looks at the ID, and decides "oh I know how to verify this", does the verification
If signature checks out there, the lawyer at the notary uses their seal to give the stamp of approval, now my letter is more powerful

makew0rld commented 2 years ago

Zion signature is an Ethereum signature. Here is info on Ethereum signature from Nit beta:

ethereum-signature-and-verification

ethereum-signature-verification-examples.zip

makew0rld commented 2 years ago

Currently blocked due to not being able to generate signatures ourselves that will validate. The demo file from Numbers does work. Waiting for Numbers to get back to us to see if they can explain/solve the issue.

makew0rld commented 2 years ago

Meeting with Numbers - May 12, 2022

Hardware sigs are broken on Starling Capture, or possibly never really worked. As mentioned above, Yurko can't generate valid software signatures on our own either. We don't know why, and Zion is no longer supported, so it'll be a ton of work to debug and get this working.

Option 1: New app
- Numbers is already working on this
- Based on Ionic
- iOS + Android
- Features like audio and CCAPI will be ported
- We need to come up with list of features we want
- Advantages: production app, well tested UX and maintained. Sigs work.
- Disadvantages: no hardware signage.
Option 2: Fix old app
- Zion may never work
- Bad UX will stay
- Zion is EOL

Numbers is checking if they can port Zion to the new app. But in general they want to move away from Zion because it is not supported by HTC anymore.

makew0rld commented 2 years ago

Using signatures generated by Yurko, Numbers confirms the Zion session key verification fails. But they say they successfully verified the AndroidOpenSSL signature, which is contrary to the results of our test. They are using their verification tool. Will test with the tool and report back.

Edit:

When using their tool it still fails to verify. This makes sense to me at least, because their tool is not doing anything fundamentally different than our test code. Bafu said he will share the code he used to verify tomorrow, so maybe there will be some clues there.

makew0rld commented 2 years ago

Numbers wants a Zion-disabled signature to investigate things further, so for now the next task is generating that.

Edit: that failed to verify as well. Waiting on their response.

makew0rld commented 2 years ago

The newer version of the capture app, 1.8.0, creates valid software signatures. Numbers says they are working on a Starling Lab fork of this version.

Edit: TODO: try 1.9.1 as they say

makew0rld commented 2 years ago

1.9.1 (starling edition) creates valid software signatures and properly sends data to our backend. Zion is still broken but Numbers says they are working on it.

makew0rld commented 2 years ago

Once https://github.com/starlinglab/integrity-preprocessor/pull/25 is merged this can be closed.

makew0rld commented 2 years ago

Update for 1.9.2, things are working now supposedly: https://hackmd.io/_BO6OtldSPata5jm4FOVBw?view#2022-08-16

[x] Test verification
[x] Update sig verification code
[x] Update schema: https://github.com/starlinglab/integrity-schema/pull/8
[x] Add negative tests
[x] Test/update HTTP API

benhylau commented 2 years ago

I cloned the app from Numbers' repo and built a modified Starling variant from develop branch.

❯❯❯ git log                                                                                                                                                                                     ✘ 1 develop ✱
commit 6f6a09dc03ca6a951bdd06143b034bac86c75be4 (HEAD -> develop, origin/develop)
Merge: 572d803 97c4381
Author: Bofu Chen (bafu) <bofu@numbersprotocol.io>
Date:   Tue Aug 16 16:47:57 2022 +0800

    Merge branch 'release-1.9.2' into develop

[...]

With the change:

-            buildConfigField("String", "STARLING_INTEGRITY_BASE_URL", "\"https://integrity-api-2.starlinglab.org\"")
+            //buildConfigField("String", "STARLING_INTEGRITY_BASE_URL", "\"https://integrity-api-2.starlinglab.org\"")
+            buildConfigField("String", "STARLING_INTEGRITY_BASE_URL", "\"http://192.168.0.214:12345\"")

And built with:

./gradlew assembleStarlingQa

Here's the custom APK build for testing: https://www.dropbox.com/s/oh9liz23qycioc7/app-starling-qa-local-for-cole-v2.apk?dl=0

makew0rld commented 2 years ago

[ ] Update HTTP API

Currently blocked on this by the app (v1.9.2) seemingly being broken and unusable, claiming permission issues after permissions have already been granted. See Slack for details.

makew0rld commented 2 years ago

Turns out the QA build is broken, but the debug one works. QA is being phased out anyway, and in the future Numbers will make a proper release build. I've built my own custom debug build to test with, and things are working! Their end doesn't seem to have changed. I've tested audio as well.

There's one part they send in the form that we don't capture, called target_provider. In my testing the value sent was Starling, and looking through the source code I can see this value is static and will be sent every time. IMO it's okay that we don't capture this, as it seems to serve the same semantic purpose as the part called tag, which also has a static value: starling-capture.

makew0rld commented 2 years ago

I've built my own custom debug build to test with

I built that with ./gradlew assembleStarlingDebug, after opening the project in Android Studio to install all the required SDKs, frameworks, deps, etc.

makew0rld commented 1 year ago

Once starlinglab/integrity-preprocessor#25 is merged this can be closed.

This is done, waiting for https://github.com/starlinglab/integrity-schema/pull/8 before closing.

starlinglab / integrity-backend