search

adobe / aquarium-fish

Your best secure distributed heterogeneous dynamic compute resource manager for CI
Other
7 stars 2 forks source link

lmplement the cluster logic #30

Open sparshev opened 1 year ago

sparshev commented 1 year ago

After #1 we've lost DQlite-based cluster formation and left with a basic implementation of the websocket-based protocol. So need to complete it to allow cluster of the nodes to work as one - what could be easier?)

sparshev commented 10 months ago

Hmmm... Thought about the cluster sync and it looks like we should not sync everything but only the necessary things. Like everything except for Applications and Votes for example for each new connection. Currently the implementation syncs only when the node is starting up, which limits the node sync after connection broken. But if we will sync only the needed things every time when connection established - this way we will have free brainsplit restoration and not that complicated logic with the update time of the cluster...

So need to think about that more... For sure will need to optimize the object storage (to not write the same objects again and again) and allow the logic to receive not existing objects from the cluster in runtime, but could worth a shot. Otherwise I don't see how to make the cluster in sync properly.

sparshev commented 10 months ago

Votes, actually, doesn't need to be stored forever - so I think we can easily remove them entirely from the DB (preventing bloat-up with not needed info) and store votes only during runtime for each NEW Application. This will need a major rewrite of the election process - which is actually good, because allow to move from pool-based to event based processing, that should improve the performance of Application processing significantly.