Implement Eth-DM

[D4nte]

Problem

See #74.

Alice wants to send a message to Bob using Waku (off-chain). Alice wants the message to be encrypted so that only Bob can read the message. Alice only knows Bob's Ethereum address.

Solution

Implement Eth-DM. Note that the protocol is not workable in its current form. The aim is to make a workable protocol and then update the protocol (tracked with https://github.com/vacp2p/rfc/issues/391).

[bgits]

First step is to find a small feature/use case where waku uses web3.js.

https://github.com/status-im/gasless-democracy

[D4nte]

First step is to find a small feature/use case where waku uses web3.js.

status-im/gasless-democracy

Could you provide a bit more details please?

[bgits]

status-im/gasless-democracy

Could you provide a bit more details please?

https://twitter.com/ethstatus/status/1293963402705018881?s=20

[D4nte]

Cool, gas-less polling!

[D4nte]

Retrieval of public key

One of the problem we are facing is that if Alice wants to send a message to Bob, and only know Bob's Ethereum Address, she does not have enough information to encrypt the message as an Ethereum address is only part of the Ethereum public key's hash.

Here are few possible solutions to consider. eth-dm public key means public key used for encryption of messages.

A. Retrieve public key from transaction

Retrieve the Ethereum public key from the blockchain by finding a transaction sent by Bob. While it seems feasible to extract the public key, one needs to crawl the blockchain to find a transaction linked to an address.

This is it not really mobile friendly except if we were to run the indexer somewhere else or use a centralised service such as etherescan

B. Bob publish eth-dm public key over waku network

Another solution would be for Bob to generate a keypair from scratch to act as eth-dm keys and publish a message to Waku containing his Ethereum address and (signed using eth key) eth-dm public key so that Alice can use the eth-dm public key.

This has several downside:

1. Bob needs to regularly re-broadcast this information as the data will not last on the Waku network (more than 30 days at the moment).
2. Alice cannot reach Bob if Bob does not broadcast first. This does not help for unsolicited messages.

C. Bob publish eth-dm public key in smart contract.

Similar to B but without B.1. drawback: the information stays in the blockchain. No need to re-publish. The downside is that you have to pay gas.

D. Alice pings Bob first

Another solution is for Alice to send a message first that just contains Bob's Ethereum address. Upon seeing this, then Bob can proceed to B (or C): broadcast his eth-dm public key over waku. Once Alice learns the key, she can encrypt the message for Bob and publish it.

E. Use Key Management network

Use a key Management network such as Torus. This involves a third party so not sure we want to go down this path.

F. Use ENS and Resolver

See https://github.com/status-im/js-waku/issues/72#issuecomment-858544401

Generation of keys

Another aspect to take in account is how Bob generates a new eth-dm private key.

I. Brand new key

Bob could generate brand new keys from entropy. In this case, Bob needs to backup the keys in a safe manner as the lost of the key would mean not being able to decrypt messages until he broadcast a new key pair (see B/C above).

II. Key from Ethereum Private Key

Another way to generate the eth-dm private key would be to generate it based on the Ethereum Private Key.

For this, one could use web3 interface to sign a salt (salt being either constant for eth-dm or something the user backs up) and use the resulting signature as the eth-dm private key. Similary to what is done with https://github.com/d1ll0n/eth-sig-encrypt/ and https://github.com/acolytec3/wakumono/

[bgits]

They can use ENS (https://ens.domains/). Bob can associate his public Ethereum and Waku key with a domain and Alice can resolve based on it.

There is already a resolver to support it: https://github.com/ensdomains/resolvers/blob/master/contracts/profiles/PubkeyResolver.sol

[D4nte]

Aiming to do B and II for now. Then will add an example or option for I. Would be good to add C, D and F at some point.

[bitsikka]

More and more projects seem to be looking into DIDs for this. Check out this discussion here. Seems like a similar problem in some aspects.

I do not exactly know how DIDs fit in here exactly but ceramic network has this: key-did-provider-secp256k1

[oskarth]

ENS seems like a good option. It is a bit problematic from a privacy POV, so doesn't always make sense.

II. Key from Ethereum Private Key

Is there more detail for how this would work? Seems useful to hook this into multiaccount design. @gravityblast might have some ideas and experience here

In general, I'd probably focus down the use case and make it minimal to suit that before generalizing. I'd also write clear user stories, as right now the solution space seems a bit too big, which to me suggests we aren't 100% clear on what exact problem we are solving.

[bgits]

It is a bit problematic from a privacy POV

What are the privacy concerns? I suppose your whisper pubkey will now be associated with your ENS name and ethereum address. Are there are concerns like now being able to identify who is communicating with whom?

[oskarth]

Same concerns as what lead to multiaccount redesign. If you can go from waku pubkey to Ethereum key you reveal transaction history etc to everyone. Compare that with a scenario where you can decide to reveal this or not, and when you reveal it you only do it to the party you are communicating with.

[staheri14]

Some thought about the concerns you mentioned in B:

Alice cannot reach Bob if Bob does not broadcast first. This does not help for unsolicited messages.

I think that is reasonable to assume that if B is part of the network and is willing to be contacted by other network participants, then he has his key published in the network.

Also wondering about the eth-dm private key generation, why Bob has to generate a new eth-dm private key? what is the use?

[D4nte]

Also wondering about the eth-dm private key generation, why Bob has to generate a new eth-dm private key? what is the use?

Web3 does not provide an encrypt/decrypt functionality nor does it provide access to the Ethereum private key.

Bob needs a private key to decrypt messages he receives. We cannot access the Ethereum private key via web3 so we need to use a new keypair.

[D4nte]

ENS seems like a good option. It is a bit problematic from a privacy POV, so doesn't always make sense.

II. Key from Ethereum Private Key

Is there more detail for how this would work? Seems useful to hook this into multiaccount design. @gravityblast might have some ideas and experience here.

Using web3 interface, sign a salt (e.g. "This is your eth-md salt; do not share the signature of this message"). The result signature (byte array) is hashed and used as your eth-dm private key.

In general, I'd probably focus down the use case and make it minimal to suit that before generalizing. I'd also write clear user stories, as right now the solution space seems a bit too big, which to me suggests we aren't 100% clear on what exact problem we are solving.

The user story got lost in translation indeed. Now updated.

[D4nte]

More and more projects seem to be looking into DIDs for this. Check out this discussion here. Seems like a similar problem in some aspects.

I do not exactly know how DIDs fit in here exactly but ceramic network has this: key-did-provider-secp256k1

Thanks for that, I agree, ideally wallets implement EIP-2884 and it would make ETH-DM a lot simpler.

[gravityblast]

Is there more detail for how this would work? Seems useful to hook this into multiaccount design. @gravityblast might have some ideas and experience here

When we designed the multiaccount feature in Status we wanted to have different keys for wallets and chats. So in the current implementation the chat key is still derived from the user's HD wallet but with a different derivation path, so chatting in a public chat doesn't reveal the address I use for my ethereum transactions. With EIP-1581 we defined a subtree of the HD wallet that should contains keys not used for ethereum (chat keys, encryption keys, etc...), and also exportable from the wallet. In Status we use this paths so that even a Keycard account is able to export a key from that subtree and keep it in memory to encrypt/decrypt messages, while all the other keys never leave the smartcard.

[D4nte]

Thank you @gravityblast. This is great info.

Eth-DM is restricted by what Web3 API gives you (whereas Status has acccess to a wallet), which means derivation is unfortunately off the table.

[oskarth]

Does that mean Web3 API doesn't support HD wallets? :o

[bitsikka]

Does it mean Browser based Status is not possible? Unless there's a version of Status Wallet that replaces Metamask?

My guess is that due to this:

Web3 does not provide an encrypt/decrypt functionality nor does it provide access to the Ethereum private key.

Ceramic Network with its DID solutions is having difficulty coming to Ethereum. They maybe having to implement their own version of Metamask that does Identity or there will be somewhat of a hackish solution similar to being pursued here.

Or, perhaps Metamask will one day support EIP-1581 and/or EIP-2884 (which seem similar in spirit).

According to this @andrewhong5297 post The Composability of Identity across Web2 and Web3, he is aiming to unify Identity/auth/wallet where similar problems surely exists.

[bgits]

Does it mean Browser based Status is not possible? Unless there's a version of Status Wallet that replaces Metamask?

It might be possible to build as a plugin using metamask snaps: https://github.com/MetaMask/metamask-snaps-beta

[andrewhong5297]

I do feel like a lot can be alleviated with snaps, I think an overhaul of the feature is coming soon. I can reach out to the Metamask team and ask about EIP-1581 implementation plans since that does seem like a cleaner path.

Alternatively, we could be approached with a DID solution like veramo messaging, though they will likely leverage ceramic and are still testing/piloting the feature. They have also tried the verifiable credential approach, allowing you to chat using an NFT (albeit not encrypted so maybe outside of the solution space). The last similar approach I can think of is with textile mailboxes. These alternative solutions do require metamask snaps to enable the storage/management of DID account and messaging apis.

[D4nte]

Does that mean Web3 API doesn't support HD wallets? :o

As far as I understand, your wallet is the one that support HD and you select in your wallet interface what account you make available via Web3.

[andrewhong5297]

On topic of EIP-1584, can you just use the wallet rpc for getting public key? Instead of using root https://docs.metamask.io/guide/rpc-api.html#eth-getencryptionpublickey

[D4nte]

On topic of EIP-1584, can you just use the wallet rpc for getting public key? Instead of using root docs.metamask.io/guide/rpc-api.html#eth-getencryptionpublickey

Oh, I see, Metamask is providing an API to encrypt and decrypt messages. Well spotted. I will have the first PoC just use web3 standard. However we could also have a PoC using Metamask API.

I see metamask proposes the usage of ethereumjs-util. I am currently using eth-crypto. I prefer to get this working first and then we can evolve the examples. My current challenges are on Waku integration, not encryption :)

[acolytec3]

@D4nte I looked at the metamask encrypt/decrypt API when I was building wakumono and it's kinda clunky as currently implemented, which is why I sent the route I did with eth-crypto. And, since it's not a commonly implemented standard, you're tying this spec specifically to the metamask ecosystem, such that it is, for the whole implementation of ETH-DM.

The real challenge here is that this standard just isn't really going to work in a seamless way with the current web3 provider spec that most wallets expose to dapps. It seems to me it works better when you bake it into a wallet implementation where the wallet already has the private key (or maybe even the seed phase) associated with the ethereum address and then encrypting/decrypting becomes trivial to implement. Building this as a messaging standard inside of the status wallet, or Trust, or Metamask or whatever other wallet might be a best case.

[bitsikka]

Interesting, I did suspect what Metamask offers or maybe headed towards offering, may not be optimal for the use cases. I do wonder how ceramic/serto and EPNS are approaching the problem. Likely less optimal, less flexible, less privacy preserving route judging by the sound of things. But maybe not.

If Andrew can steer the way in Consensys domain. That would be awesome! :)

Learning a lot - thank you all. A lot for me to go look and chew on as well.

[D4nte]

Thank you everyone for the great discussion. I implement a first version of Eth-DM in #204.

The aim of Eth-DM is to demonstrate application that could be built with js-waku. Do feel free to continue the discussion on the spec side: https://github.com/vacp2p/rfc/pull/403 or in the follow-up issue: https://github.com/status-im/js-waku/issues/211.