Version the protocol?

[noahlevenson]

What's our strategy for enforcing compatibility in a world of protocol changes? There can be a simple and suboptimal approach today, with a better approach to follow later.

[noahlevenson]

Here's my thinking, LONG TERM and SHORT TERM:

SHORT TERM Today, every peer must contact our discovery server to participate in the network. So it seems reasonable that the discovery server should enforce version compliance. Peers can include an x-broflake-protocol-version header with their HTTP requests to Freddie, and if Freddie thinks you're running an incompatible version, he'll send you back a response letting you know.

For users who have managed to run an outdated and incompatible client, they'll discover it the next time they attempt to fill an unused connection slot. In other words, peers will get tested for version compliance many times per day through the natural process of network churn.

LONG TERM We probably won't always have a centralized discovery server, which means that the above strategy has an expiration date. Moreover, I think we want to enable a world where v1 and v2 peers can coexist on the network, as long as v1 peers only speak to v1 peers and v2 peers only speak to v2 peers.

This is pretty standard stuff for p2p networks. The discovery mechanism allows anyone to advertise themselves on the network, and it's up to you to vet the protocol version of any peers who make you an offer.

To make this work, we'll need to add a handshake. Adding a Broflake handshake has been discussed in other contexts for various purposes, so it seems like this is inevitable.

[noahlevenson]

More of my rambling from Slack:

thinking more deeply about the versioning discussion -- and maybe I should add this to the GH thread -- but I think we're reaching a crossroads with respect to our monorepo.

here's the challenge, i think:

Broflake is a p2p protocol which we hope will be interoperable and useful for lots of other people. so when we bump a major version that isn't backwards compatible, it has serious cascading effects. you can imagine, if we had several partners running Broflake-compatible clients on the Lantern Network, the disruption that would ensue when suddenly we release a new v2 client that doesn't want to play with all the v1 clients on the network.

well, our UI is where our application-layer special sauce lives -- it's the part of our stack that isn't really interoperable. so i think a question worth asking is: to what extent should changes to our UI have any impact on the rest of the Broflake stack?

to make it more concrete: if the UI gets a patch, should Broflake get bumped from 1.0.1 to 1.0.2? and if so -- why should our partners at Signal or wherever now have to deal with a version change which actually didn't alter a single character in the code they're running?

one solution could be to semantically version each part of the Broflake stack independently -- that is, we could be in a world where the egress server is v1.0.4, freddie is v2.0.6, the client engine is v2.1.1, and the UI is v7.6.4.

this sounds like a nightmare already, though. just keeping track of which versions of each component are compatible with each other seems... impossible.

IMO, it probably makes more sense to decouple the protocol layer from the application layer, and just keep track of two versions: the protocol version, and the UI version?

[noahlevenson]

After more chats, some lemmas:

1. Build time versioning doesn't really make sense, because some pieces of Broflake occasionally get built as libraries by a controlling process. Why should it be the responsibility of Flashlight to inject a protocol version for Broflake?

2. Each software component in the monorepo is actually implementing a protocol spec in a whitepaper that we just haven't written yet. So there are really two notions of version: the version of the concrete implementation, and the version of the protocol contract that the implementation abides by.

3. In practice, when we talk about enforcing version compatibility, we're talking about enforcing the protocol version, not the version of the concrete implementation. Freddie just wants to make sure you're speaking the right protocol on the network. He shouldn't care if your client is two minor versions behind, as long as you still respect all the contracts of the current protocol version. (This is the desired logic when peers eventually enforce version compliance in a decentralized fashion via some kind of handshake.)

4. As Broflake's developers, it seems useful to be able to interrogate what version of the concrete implementation our users are running.

5. The solution is probably just semantic versioning, where the major version is the protocol version. All the Go code in the monorepo -- as Broflake's official reference implementation -- should share the same semver. For implementers, all they need to know is that the major version represents protocol compatibility. So if someone else writes a Broflake client based on our reference implementation at 1.4.1, that client should have no problem interacting with other v1 peers, even in a world where there are no longer centralized services to vet compatibility.

6. Right now, we'll program Freddie to enforce compatibility on the major version. So if we bump Broflake from 2.0.7 to 2.1, each component in the monorepo will now be considered version 2.1. Freddie will allow everyone >= 2.0.0 to participate.

7. We just need to be very careful to make sure we update the hardcoded version in common when we're supposed to!

[noahlevenson]

The basics are implemented here:

https://github.com/getlantern/broflake/pull/117