Data encryption

[alexcos20]

Some use cases require data to be encrypted, especially when stored on public decentralized infrastructure like IPFS or Arweave (because anyone can read it).

Let's fix that:

• users uploads data to DBS (Decentralized Backend Storage)
• if encryption is required, DBS encrypts the file using provider
• DBS uploads file (encrypted or not) to Arweave/IPFS
• User gets ipfs hash/arweave tx, etc and published that
• On consumption, provider detects if the file is encrypted. If not, it's served to the consumer. If yet, it is first decrypted in memory and then served to consumer

[trentmc]

Right now people can share encrypted data. Just that encryption & decryption has to be done client-side. And that's ok for many use cases! We shouldn't ignore this.

I worry about Provider doing yet more work. It means that people have to trust the Provider runner that much more; and it means more complexity in the Ocean stack. I'd prefer to avoid both, if possible.

There are many possible flows on encrypted data. For a specific flow, one needs to work out:

1. is encryption / decryption symmetric or asymmetric?
   • if symmetric, how is the encryption key shared?
   • if asymmetric, how does the encrypting party get the public key to encrypt in?
2. where is the data? It could be (a) on-chain via metadata / Aquarius, (b) on-chain via ERC725, (c) on-chain via urls (eg Arweave), or (d) off-chain via urls (eg Filecoin, web2 storage, ..).
3. is the encryption in GUI, JS, or Py? Is the decryption in GUI, JS, or Py?

The ocean.py README "Private Sharing of On-Chain Data" has examples for (1) both symmetric & asymmetric encryption, (2) for when data is on-chain via ERC725, (c) both encryption & decryption in Py

• Has symmetric encryption to share the main data blob
• Has asymmetric encryption to securely share the symmetric private key over a public channel
• This combo of symmetric & asymmetric is a common tactic in cryptography. That way the main data blob is only stored once, yet you can share to many people.
• The example doesn't add heavy new functionality to Ocean stack . Rather, it leverages simple-to-use methods in ocean.py to do the following.
  • (i) store & retrieve arbitrary data (encrypted or decrypted)
  • (ii) create useful symmetric key
  • (iii) symmetric encrypt & decrypt
  • (iv) asymmetric encrypt & decrypt
• Where (i) is in Datatoken.py, and (ii)(iii)(iv) are in a small new module crypto.py that wraps 3rd-party crypto libraries for a simple unified interface
• These building blocks can be composed in arbitrary ways, allowing for a variety of flows

So if any new work is done for this issue, I recommend that we identify which specific flow we'd like to support better. That is: answer the questions (1)(2)(3) above.

• Perhaps for some flows, what Alex has suggested is the best
• For other flows, it might be where ocean.js gets new simple-to-use crypto methods for (ii), (iii), (iv) like crypto.py

[alexcos20]

@trentmc - your flows are only working when you know who is buying the asset. But this is a very very narrow case. For a regular publisher, this does not apply, because he/she does not know in advance who will buy the asset.

As publisher, I have the following concerns:

• somebody can discover & fetch my ipfs CID and access my dataset
• somebody can discover & fetch my Arweave publisher address and fetch all files published by looking at my txs.

• somebody will discover my url using some http crawlers

  How can a publisher increase security:

  • if dataset is stored on-prem, use firewall and only allow provider to fetch it
  • if using API or similar, and firewall is not an option, add an authorization header (which is encrypted by provider) and allow content fetch only if that header is passed to the request
  • encrypt data

Possible ways to encrypt when buyer is not known:

• use provider and trust it (proposed flow above)
• encrypt data before publish and share the key with the buyer. But this flow has a lot of drawbacks. After data is bought, publisher needs to send the decryption key to the buyer
  • additional latency: what if publisher is on holiday ?
  • additional costs: share the key on-chain costs gas
  • what if publisher refuses to send the key? Buyer paid for nothing, he is not able to decrypt

[alexcos20]

The ideal case would be to have this sharing key mechanism in datatoken contract. IE:

• data is encrypted before publishing
• datatoken is updated to hold the decryption key (symmetric)
• when an order is confirmed, decryption key is encrypted using buyer public key and available in Order event.
  • buyer fetches that, decrypt event, gets the decrypt key and then decrypts the file

Unfortunately, in solidity, I don't think is possible for now. Maybe a combination of on-chain/off-chain components can do the trick, but this is still in a design phase

[trentmc]

ideal case would be to have this key sharing mechanism in datatoken contract

Actually you can do it with threshold cryptography. Eg Lit Protocol. The key gets sharded up across nodes in Lit Network, then reconstructed for the user when needed.

I bet you could prototype this in a matter of hours

• for Lit, you have to define the on-chain conditions that must be met in order to be able to retrieve the key
• For Ocean, that condition would be "has performed an order operation"
• Dig into this into the Lit docs, you'll see what I mean within 15 min of reading

your flows are only working when you know who is buying the asset. But this is a very very narrow use case

Actually the flows work across a super broad spectrum. Example uses:

• for any dapp to store private user data. This includes the large category of people building dapps with Ocean.
• anything where the sender knows who the receiver is. Such as every messaging app, ever: email, whatsapp, slack. And data sharing like Google Drive permissions or Dropbox
• OTC data sales
• and more

While broad, the above cases do not support the "data marketplace" use case, where buyer doesn't know seller (as you point out). It would be useful to support that flow, and the broader version of that flow which is all around datatokens.

Ideally we support that via Lit. (Perhaps this is the perfect time to introduce a third datatoken template that does exactly this. Not only for encrypt / decrypt of data, but of url itself. Two birds one stone. I believe it's less work than one think.)

The other option to support this flow would be like you suggested in the description - to entrust Provider with the keys to encrypt / decrypt.

My recommendation is to try the Lit approach first. It's more trustless, and where we want to be eventual anyway.