This must be coded as a replication feature, that's what it is.
Today Minka relies on the client application storage, thru the PartitionMaster interfase implementation.
Which is recommended to be in a distributed database, or better said, in carge of the host app.
That's not quite a highly-available with fault-tolerant solution, If I let the restore of duties to become from the application instead of being able to provide it from Minka.
All the behaviour today is HA and FT, but to achieve a storage by my own, would turn the equation much more seriously.
This's will mean to cluster-synchronize -for example as the hadoop replication- the reception of CRUD operations over duties, throughout all the shards in position to be an electable leader.
I imagine something really easy to achive, like telling all the followers to prepare to receive and ack data, in an atomic TX.
The persistence could be in the filesystem.
Lets understand, that duty payloads are data used only by the task executor taking the duty.
This doesnt include other situations like having to read some distributed filesystem or storage where the data to process relies, thats completely at the side of the host application, in such cases the duty payload may be only a string representing a filepath, etc.
Use www.Atomix.io CopyCat library or any other RAFT concensus algorithm implementation
This must be coded as a replication feature, that's what it is. Today Minka relies on the client application storage, thru the PartitionMaster interfase implementation. Which is recommended to be in a distributed database, or better said, in carge of the host app. That's not quite a highly-available with fault-tolerant solution, If I let the restore of duties to become from the application instead of being able to provide it from Minka. All the behaviour today is HA and FT, but to achieve a storage by my own, would turn the equation much more seriously. This's will mean to cluster-synchronize -for example as the hadoop replication- the reception of CRUD operations over duties, throughout all the shards in position to be an electable leader. I imagine something really easy to achive, like telling all the followers to prepare to receive and ack data, in an atomic TX. The persistence could be in the filesystem. Lets understand, that duty payloads are data used only by the task executor taking the duty. This doesnt include other situations like having to read some distributed filesystem or storage where the data to process relies, thats completely at the side of the host application, in such cases the duty payload may be only a string representing a filepath, etc.
Use www.Atomix.io CopyCat library or any other RAFT concensus algorithm implementation