Proofmode Integration

[foundscapes]

What needs to be done for us to do this?

[foundscapes]

@n8fr8 handing to you since you mentioned other funding to support this

[foundscapes]

Would like information on where proofmode integration w/ ios stands

[n8fr8]

We have a library nearly ready that we can work with @tladesignz on how to integrate.

We also have a greatly-updated version of the Android library (1.0.23+ release) that we should talk about using now in Save.

Finally, we have a public ProofCheck tool that makes verification of proof bundles easier: https://proofcheck.gpfs.link/ https://proofmode.org/verify

[foundscapes]

Wonderful news!

[hiromipaw]

@n8fr8 seems like proofmode for android is still using a static password for the local key store. We would like to be able to pass a user or app generated string. I wonder if that's something that is already in your roadmap or maybe we should send a PR your way.

[n8fr8]

I think we have that in our roadmap for the library - both allowing a developer to provide the password, and possibly the key, as well. https://gitlab.com/guardianproject/proofmode/proofmode-android/-/issues/16

[tladesignz]

@hiromipaw, please note, this is an issue of Save >iOS<.

@n8fr8, can you ping me, when Proofmode iOS is ready? Thank you!

[hiromipaw]

@tladesignz apologies for the confusion, I should have specified, that my question was more about not having the same issue on iOS as well.

[tladesignz]

Ok, so I checked out the current state of ProofMode iOS:

The latest code can be found here: https://gitlab.com/guardianproject/proofmode/proofmode-ios/-/tree/main_npex

I cannot currently build it, because I don't understand the build instructions. ("gomobile-iofs" does not exist. GoMobile is typically used the other way round, so I'm at a loss here. Anyway, not so important right now, I could find out.)

It's currently not in a state of a reusable library. Instead, it's an app and the important files which would be needed are scattered around the project.

Most important file seems to be this one: https://gitlab.com/guardianproject/proofmode/proofmode-ios/-/blob/main_npex/Proofmode/Proof.swift

Anyway, this doesn't look like it's going to be in a usable shape any time soon. I estimate at least half a year.

[n8fr8]

@tladesignz please join our ProofMode scrum tomorrow morning, and you can talk with n-pex and I about it directly. He has been beginning the work on the libProofMode library for iOS, and it may not all be in the places you expect.

[n8fr8]

The current testflight release of the app itself (which is in a very usable state) is here: https://testflight.apple.com/join/X2cdrTrK

as linked to from here: https://proofmode.org/project/proofmode-ios

[tladesignz]

Proof check: https://proofcheck.gpfs.link

[tladesignz]

Ok, so n-pex created a lib for me. Turned out, it was already half-done on his computer: https://gitlab.com/guardianproject/proofmode/libproofmode-ios

I have some complaints he wants to address tomorrow.

Turns out, the whole thing is actually not that big, so I can always fork and rework it to our needs.

[n8fr8]

Fork?! Please upstream and improve, as the ProofMode JSON schema/output will continue to evolve, and it would be great to have your implementation receive this improvements.

[tladesignz]

Happy to upstream. Fork is just last resort in case you don't move in the direction we need. Currently lib is not abstract enough for reuse. Still too many assumptions from your own app in there. And some changes I might need might break API, but I won't fix your Proofmode app. (If GP isn't paying for it, that is...)

[tladesignz]

Ok, so I recently played around with this and got my first three patches merged.

It can now compile with Save and we only need to drop support for iOS 12.

I'm still pretty unhappy with the overall design, though. Unfortunately, it was modelled after the Android version and also in a first-app-then-we-fork-out-a-lib fashion. Therefore it has a lot of assumptions on where things should be, esp. files.

I will need to work more with Mikael to make things work. It currently contains its own multi-threaded proof processor, which makes it nearly impossible to find out the names of the generated files which depend upon the content hash (a very expensive operation on videos!)

It also has all the same problems with key handling as the Android version. (See https://github.com/OpenArchive/OA-Trail-of-Bits/issues/16)

Also, I'm unhappy with how it is implemented in Save Android: The proof is only generated straight before the first upload attempt. I instead want to generate proof right when the files are added, instead. This has the upside that the proof timestamps are from when the files where added to the app, so hopefully way closer to when the assets where created, and it doesn't make the upload wait on the proof generation.

[foundscapes]

Thank you for this update @tladesignz. It sounds like we'll need a bit more time and work before it's ready. We also want to ensure that it's working well when people do in-app capture, which would be the ideal way to use it to capture the most sensor data right?

[tladesignz]

Thank you for this update @tladesignz. It sounds like we'll need a bit more time and work before it's ready.

Yep. It's not a big thing. But I would want to have a decision about how to do the key handling, and, depending on that, there is some UX work involved and more or less work on the library we either need to wait for or contribute to.

We also want to ensure that it's working well when people do in-app capture, which would be the ideal way to use it to capture the most sensor data right?

I guess "working well" in that regard would mean that the proof is generated right after capture? It wouldn't be about capturing "more" sensor data, but about capturing it closer to the image, which makes the proof more meaningful, since that says that the device, where the proof was generated and the device, where the image was captured, are the same.

BTW: compass/GPS data and GSM network data doesn't get used in the config Save Android uses currently.

[tladesignz]

Ok, so I have it working, mostly.

Remaining issues:

• Waiting on merge of pull requests: https://gitlab.com/guardianproject/proofmode/libproofmode-ios/-/merge_requests

• Public key export still missing.

• Can't make it work on Internet Archive. It seems, its just taking the first file sent and then stalling me endlessly. @n8fr8, it seems to work on Android, but it creates a new "item" which contains all proof files. Can you explain, what happens and what's the difference?

Here's my code (proof data currently disabled): https://github.com/OpenArchive/Save-app-ios/blob/master/Save/Management/IaConduit.swift

Here's the current Android code: https://github.com/OpenArchive/Save-app-android/blob/master/app/src/main/java/io/scal/secureshareui/controller/ArchiveSiteController.kt#L115

• The signatures are worthless without the public key. How do we deal with that?

It doesn't make sense to send it to the server per asset. It always stays the same.

The Android code doesn't seem to send it to the server, either.

It might make sense to send it once per collection. However, that's going to be really ugly to implement, as there's no concept of a collection upload, only an asset upload. Checking, if the file is already there, per asset also seems ugly, but slightly better.

@n8fr8, maybe you can clarify: what was the original idea of how the public key gets passed around? Just by exporting it and sending it via another channel?

[n8fr8]

Yes, the original idea was that the user would just export and distribute their public key however they saw fit. They might even cross-sign it with a known public identity, or add it to keybase or some other system like that. We should still give them that freedom.

With the current ProofMode official apps, we generate a proof bundle zip file that has all the various proofmode generated files (csv, json, asc, optional 3rd party notary data, public signing key). The proof zip also contains the photo or video. This is meant as a full self-contained atomic bundle of media+proof+identity that can be shared through messaging apps, sneakernet'd over bluetooth or airdrop, or posted to IPFS, etc.

This proof zip bundle is also what the ProofCheck tool https://proofcheck.gpfs.link/ and underlying scripts expect for easy verification. Of course, manual verificatino of proof is always possible, as documented here: https://proofmode.org/verify

YES, we include the public key in every proof zip bundle, because again, we want this to be self-contained, and also there is still no good, open, reliable openpgp public key server, and proofmode itself does not have any centralized infrastructure for identity. It is also a tiny bit of data, so it really doesn't matter from a bandwidth or processing perspective.

What I would like to see is a proof zip generated with the pubkey.asc, but without the media file in it. It would be uploaded along side the media file. We are enhancing the ProofCheck tool so that you can easily add both the proof zip and corresponding media file to do the verification easily.

If this all seems too much and not compatible with what you've implemented, then fine, feel free to put the user's proofmode public key where you want in your collection metadata. It doesn't matter to us ultimately, as long as it exists somewhere for use in manual verification as needed down the road.

[tladesignz]

Ok, so added public key export and upload.

Internet Archive still not working. No idea, why. It takes the first file which finishes, and seems to ignore the rest.

And I don't really understand the difference to the Android code, besides, that the Android version actually creates 2 items: The actual asset, and another item for all the metadata.

[foundscapes]

Thank you for this update @tladesignz. It sounds like we'll need a bit more time and work before it's ready.

Yep. It's not a big thing. But I would want to have a decision about how to do the key handling, and, depending on that, there is some UX work involved and more or less work on the library we either need to wait for or contribute to.

did this get resolved?

We also want to ensure that it's working well when people do in-app capture, which would be the ideal way to use it to capture the most sensor data right?

I guess "working well" in that regard would mean that the proof is generated right after capture? It wouldn't be about capturing "more" sensor data, but about capturing it closer to the image, which makes the proof more meaningful, since that says that the device, where the proof was generated and the device, where the image was captured, are the same.

Isn't the primary value of Proofmode that it captures relevant sensor data at the time of media capture, not after? Our users need it to work this way to capture cell tower data + barometric + etc. etc. to corroborate exif data. We need to make sure this is clear to users as well - perhaps with a popup saying: "Proofmode is most effective when you capture your media in-app." or something similiar.

BTW: compass/GPS data and GSM network data doesn't get used in the config Save Android uses currently.

We would want to ensure it does, and also bring back in-app capture to best utilize Proofmode. Will make an android issue.

[foundscapes]

Ok, so added public key export and upload.

Internet Archive still not working. No idea, why. It takes the first file which finishes, and seems to ignore the rest.

And I don't really understand the difference to the Android code, besides, that the Android version actually creates 2 items: The actual asset, and another item for all the metadata.

We need to figure this out...let us know how we can help

[tladesignz]

I think it's super-ugly to create another asset on the Internet Archive, which is actually just the ProofMode metadata of another one.

This definitely should go into the container, each asset gets on the Internet Archive anyway.

You would need to clarify with IA, how and if that is possible.

On the other hand, I think with ProofMode and the new C2PA stuff, there's going to be metadata attached directly to the content file. (Although, that's never going to be true for every file format.) So, that could also be a way out.

That would need to be clarified with @n8fr8.

[n8fr8]

I think this requires a call. I agree, the new integration with C2PA does offer a way forward here that fits more cleanly within the existing Save model.

[foundscapes]

Ok let's set up a call @n8fr8 @tladesignz to discuss this and a few other issues asap, I will coordinate!