search

stacks-network / sbtc

Repo containing sbtc
GNU General Public License v3.0
206 stars 2 forks source link

[Feature]: Use libp2p for signer communication #409

Closed djordon closed 5 days ago

djordon commented 1 month ago

Feature - Use libp2p for signer communication

1. Description

We should configure the sBTC signers to use libp2p for communication. This would increase our resilience if the relay server fails.

The signers would be structured as follows:

  1. Use a clarity contract as a central data source for the public keys of the signers. Each signer can query their stacks node for the current-signer-set data variable in the sbtc-registry contract. This is just an API call to the stacks node to retrieve the current-signer-set, not a contract call.
  2. We could setup a relay server as before, but its role is to facilitate establishing direct connections with peers that are behind a NAT.
  3. Have each sBTC signer use a libp2p GossipSub messaging protocol with authentication, using the private key that corresponds to the known public keys.
  4. Use connection gating to ensure that only peers from the address noted in the smart contract are valid. In libp2p, peers are identified by their PeerID, and this identifier is set by their public key.

1.1 Context & Purpose

This involves pulling the libp2p crate into the signer binary and configuring it for signer to signer communication.

2. Technical Details:

There are at least two hard parts here:

  1. Implementing connection gating. I believe this requires an implementation of the libp2p::swarm::NetworkBehaviour trait.
  2. Managing connection changes when we rotate keys. When we rotate keys, signers will probably start signing messages with the new key. It seems dangerous to keep signing messages over existing connections using an "old" key. This is something that we need to handle using the current design.

2.1 Acceptance Criteria:

For this ticket, we do not need to support establishing connections with servers behind a NAT.

3. Related Issues and Pull Requests (optional):

djordon commented 1 month ago

Use of libp2p

The libp2p seems to be a very mature and actively maintained library.

  1. The library is created and primarily maintained by Protocol Labs, which are the same folks behind IPFS and Filecoin.
  2. Ethereum 2.0 (post The Merge) uses libp2p in its consensus layer. I believe they use the go implementation of libp2p.
  3. The polkadot blockchain https://github.com/paritytech/polkadot-sdk uses the Rust implementation of libp2p.
  4. It is audited, although I cannot find some of the evaluation reports. https://ipfs.io/ipfs/QmRAFP5DBnvNjdYSbWhEhVRJJDFCLpPyvew5GwCCB4VxM4.

Useful links

cylewitruk commented 1 month ago

What's the current thinking around signer peer discovery? I see neither a configuration for providing bootstrap/seed nodes, nor that addresses are stored in the sbtc-registry contract (only keys).

Spontaneously I think that storing signer addresses in the contract next to their pubkeys would be the most optimal, both from a network-resilience perspective (avoid central nodes) and from a security perspective (can filter incoming connections to valid signer peers).

Am I missing something or is this something that hasn't been decided yet?

djordon commented 1 month ago

What's the current thinking around signer peer discovery? I see neither a configuration for providing bootstrap/seed nodes, nor that addresses are stored in the sbtc-registry contract (only keys). ... Am I missing something or is this something that hasn't been decided yet?

You're not missing anything, that stuff hasn't been decided yet.

Spontaneously I think that storing signer addresses in the contract next to their pubkeys would be the most optimal, both from a network-resilience perspective (avoid central nodes) and from a security perspective (can filter incoming connections to valid signer peers).

Hmmm, I haven't thought of that. I was thinking that a few seed peer addresses would be "known" and stored in all of the signers' configuration files. Keeping them in the contract seems like a good idea too, but requires that each signer know their own IP address at runtime. I think that might exclude certain network setups, but these network setups might be excluded anyway (I don't think we can have a signer in a private subnet behind a NAT due to our basic use of libp2p, but I could be wrong here. Also, maybe we could use more advanced features of libp2p and get around this but we still require a signer to be able to figure out their IP).

I think it would be really good to get some more eyes on this question.

djordon commented 1 month ago

What's the current thinking around signer peer discovery? I see neither a configuration for providing bootstrap/seed nodes, nor that addresses are stored in the sbtc-registry contract (only keys).

I would proceed by assuming the path forward is putting few of them in a config file. If we go some other route, like the smart contract route, then that would require us to change the smart contracts and do other a bunch of other work. We could go some other route here but I think we could get most of the core libp2p logic done without finalizing this decision right now.

cylewitruk commented 1 month ago

@djordon sounds good, will open up for seed peers to be placed in the config :+1:

cylewitruk commented 2 weeks ago

@djordon when it comes to this:

Use connection gating to ensure that only peers from the address noted in the smart contract are valid. In libp2p, peers are identified by their PeerID, and this identifier is set by their public key.

What constitutes "valid" in this case? Peers which aren't part of the active signer set surely need to have the ability to communicate with the network in order to be eligible for becoming an active signer? If so, then which messages should not be gated?

djordon commented 2 weeks ago

What constitutes "valid" in this case? Peers which aren't part of the active signer set surely need to have the ability to communicate with the network in order to be eligible for becoming an active signer? If so, then which messages should not be gated?

Good question. The idea is to use the fact that only those who have made the appropriate PoX smart contract call are eligible to be signers. So only those addresses where the signer has a record of the relevant PoX smart contract call are "valid". We only have their stacks address in this case, not their public key. But thinking through this some more, it's clear that this is awfully complex. Maybe there is a simpler way?