search

dotmesh-io / dotmesh

dotmesh (dm) is like git for your data volumes (databases, files etc) in Docker and Kubernetes
https://dotmesh.com
Apache License 2.0
539 stars 29 forks source link

[0.5d] Extend commit notifications to survive DM restarts and work in multi-node clusters #587

Closed alaric-dotmesh closed 6 years ago

alaric-dotmesh commented 6 years ago

Issue #574 gave us a mechanism for registering to receive commit notifications from a Dotmesh server - a single node cluster.

It's not great for a multi-node cluster, as the subscription only exists on one node. That means any unsubscribe RPC calls need to be routed to that server alone (tricky in a kubernetes cluster), and if that node goes down, the subscription is lost.

My original plan for multi-node operations was to put subscriptions in etcd (although it greatly pained me that this would mean keeping a record of the NATS password there), and having every node responsible for sending out notifications about commits registered on that node; but @rusenask suggests that, if we're not planning on developing Dotmesh except in ways demanded by Dotscience, we're not going to need multi-user subscriptions (or subscriptions other than commit notifications), so we can drop the general subscription machinery and just have an environment variable, checked at dotmesh-server startup, pointing to a nats subject to send notifications of commits to.

So: Decide if we want to go general and have a registry of subscriptions in etcd, managed dynamically through the RPCs I added for #574; or go simple and have a single static subscription created at startup if the env var is present! Then do it!

rusenask commented 6 years ago

TODO: refactor integration test for this. Currently moved test into the unit tests on the dotmesh server but it would be good to have another one in the integration tests.