The basic idea is to insert an AWS ALB in front of the server and then have a deploy consist of spinning up a new mb2 container, where the first thing that is transferred is some new lobby connections, then the smallest event is transferred, then the remaining lobby, then the remaining events, then the old container is shutdown.
I haven't worked out all the details, but the ALB should be programmable via rusoto_elbv2. Since the web-socket url is not (normally) visible to the user, my inclination is to add some sort of routing identifier (perhaps a sequential id, perhaps a UUID, perhaps a digest of the binary) to the web-socket url in such a way that would allow us to rollback trivially.
On the database side, we'd sometimes have to do multi-step migrations that initially provide a backwardly compatible schema, with the non-backwardly compatible change only being made after everything has been running fine with the backwardly compatible schema for "a while".
The protocol side will require some investigation. I don't think cbor_serde gives us a guarantee that we can introduce a backwardly compatible protocol, which would mean we'd have to have some minimal downtime each time we changed the protocol in a non-backwardly compatible manner, however, we'll want to minimize protocol changes anyway once we start developing native clients.
I'm tagging this with KickStarter, because we're bound to have a bunch of awkward crashes when I initially implement this, but afterward things should be much more robust, so it'd be good to get this done before greatly expanding the number of people using or observing the software.
Make it so most deploys require no downtime.
The basic idea is to insert an AWS ALB in front of the server and then have a deploy consist of spinning up a new mb2 container, where the first thing that is transferred is some new lobby connections, then the smallest event is transferred, then the remaining lobby, then the remaining events, then the old container is shutdown.
I haven't worked out all the details, but the ALB should be programmable via rusoto_elbv2. Since the web-socket url is not (normally) visible to the user, my inclination is to add some sort of routing identifier (perhaps a sequential id, perhaps a UUID, perhaps a digest of the binary) to the web-socket url in such a way that would allow us to rollback trivially.
On the database side, we'd sometimes have to do multi-step migrations that initially provide a backwardly compatible schema, with the non-backwardly compatible change only being made after everything has been running fine with the backwardly compatible schema for "a while".
The protocol side will require some investigation. I don't think
cbor_serde
gives us a guarantee that we can introduce a backwardly compatible protocol, which would mean we'd have to have some minimal downtime each time we changed the protocol in a non-backwardly compatible manner, however, we'll want to minimize protocol changes anyway once we start developing native clients.I'm tagging this with KickStarter, because we're bound to have a bunch of awkward crashes when I initially implement this, but afterward things should be much more robust, so it'd be good to get this done before greatly expanding the number of people using or observing the software.