New transport: BOLT-8 or ZeroMQ?

[ecdsa]

We have been considering BOLT-8 as a replacement for SSL in client-server communications. Another option could be to use ZeroMQ. Pros:

• high performance. This might allow us to deploy privacy improvement such as #496 or #4439.

Cons:

• adds a new dependency.

I do not know what kind of performance improvement can be expected over SSL or BOLT-8. @evoskuil, maybe you can comment on that?

[evoskuil]

BOLT-8 is a handshake protocol, intended to provide authentication and privacy. ZeroMQ is a transport independent concurrency framework. The ZeroMQ library incorporates the handshake aspect of DJB’s CurveCP, which is the proper analog to BOLT-8. While SSL deserves replacement for a number of reasons, like cryptography, rolling your own is generally ill-advised.

We have integrated a good amount of the capabilities of ZeroMQ. This includes its integrated SOCKS5 support (required for TOR) and CurveCP (for server and optionally client authentication). I can attest to the simplicity of these integrations. There is no more than a configuration setting for each.

CurveZMQ requires a 256 bit key pair, with a private key for proving server and optionally client identity. Public keys at the opposite ends for verification of that identity. Key generation is a simple algorithm. We implemented a keygen command pair in BX and incorporated keys into client and server config files for BS.

Performance comparison can be a complex subject, but the good folks at ZeroMQ provide quite a bit of performance analysis. And the CurveCP handshake is as trivial as it could be. It was designed for simplicity (as a central aspect of security) and from my reading it shows about 1/3 to 1/4 of the byte overhead of SSL.

The choice of ZeroMQ is a larger question than choice of handshake protocol. ZeroMQ is as light a framework as you could have while not reproducing it yourself. It deals elegantly with the many complex aspects of distributed systems. When rolling your own, one learns the many lessons built into the design of ZeroMQ over time.

Libbitcoin takes dependencies very seriously. I’ve discarded maybe a dozen of them. Our base libraries only have libsecp256k1 and Boost (practically essential for c++). Our client-server libraries only add the ZeroMQ dependency, as it would be insane to roll this ourselves which achieving the benefits it provides.

ZeroMQ has a robust and long-standing open source community, as well as broad corporate support. But the decision to use it is not simply about the curve protocol option, or even its outstanding performance. It’s a distributed system design decision. It provides robust capability with little coding effort to implement rational failover, load-balancing, broadcast and scale patterns. No need for load balancers, no need to code in hacks when clients are either too slow or too fast, no need to bolt on security or proxy protocols, no need to code changes to support various network protocols. The same code works in-process, out-of-process, LAN, and Internet, with corresponding automatic performance optimization.

The only downside is that it’s not http, which is also its upside.

[SomberNight]

BOLT-8 is a handshake protocol, intended to provide authentication and privacy

BOLT-8 is not just about a handshake; you can put it on top of TCP and you have a full transport. See e.g. how the message format is specified here.

The current protocol uses: JSON over TLS/SSL over TCP This could be replaced with: JSON over BOLT-8 over TCP That is, it is ~cleanly replacing SSL.

Libbitcoin takes dependencies very seriously. I’ve discarded maybe a dozen of them. Our base libraries only have libsecp256k1 and Boost (practically essential for c++). Our client-server libraries only add the ZeroMQ dependency

We already have BOLT-8 implemented as we need it for lightning. It is not a lot of code either. In terms of dependencies, besides the python stdlib, for chacha20-poly1305 it needs either python-cryptography or pycryptodomex. On the client side, we already have and use it though, so it would only be a new requirement on the server side.

While SSL deserves replacement for a number of reasons, like cryptography, rolling your own is generally ill-advised.

Reusing the same transport we already use for Lightning for the electrum protocol is not really "rolling our own" encryption.

---

Note that one issue with using BOLT-8 is that it has a max message size of ~65 kbyte, as 2 bytes are used for message length, which is too small for some messages in the electrum protocol (e.g. block header chunks, addresses with large histories). This means we would either have to

• modify it a tiny bit to allow larger messages so that our JSON payloads can fit into a single message, or
• somehow allow splitting payloads over several BOLT-8 messages

[evoskuil]

I’m not referring to you rolling your own, but to BOLT-8 doing so. Already requiring it is of course an important dependency consideration. TCP is already a full transport, and as I understand it BOLT-8 is providing a TLS replacement. The handshake provides for key exchange, just as with TLS and with CurveCP.

However the choice of ZeroMQ is largely orthogonal. As I mentioned, it’s not a transport, it’s a concurrency framework for high scale distributed systems. TCP, TLS, and BOLT-8 are not comparable (and of course neither is JSON).

ZeroMQ transports transparently over many protocols, including TCP, and provides the security and privacy aspects of CurveCP as a TLS replacement. And of course JSON can be carried over any transport. But the purpose of ZeroMQ is to make exceedingly simple the common patterns of distributed systems, which is far more involved than simply opening a socket.