adobe / aquarium-fish

Your best secure distributed heterogeneous dynamic compute resource manager for CI
Other
7 stars 3 forks source link

Solve the DQlite issues #1

Closed sparshev closed 2 years ago

sparshev commented 2 years ago

In general DQLite was a good start for the project, but it seems we need to move on. There is a number of issues with using DQLite:

So I think it will be better to switch to using sqlite for the node with a custom cluster communication protocol. The info distribution could be done via websocket and the nodes are establishing those connections to a limited number of neighbors to properly auth in the cluster via certificate and sign their requests to verify based on pubkey.

There is still a question how vote process will look like, and how the future web interface will collect the cluster information, but probably it's doable.

Expected Behaviour

DQLite need to be removed and replaced by sqlite and custom websocket-based cluster communication protocol.

Actual Behaviour

For now the Fish uses a number of C-based dependencies including DQLite

sparshev commented 2 years ago

Ok this ticket is finally closed, the only issue was not resolved is cluster sync - but since it not worked well in dqlite than it's a separated one #30.