search

danthegoodman1 / UltraQueue

Distributed pull queue based on Segment's Centrifuge. Zero dependency p2p clustering.
GNU Affero General Public License v3.0
1 stars 0 forks source link

ScyllaDB Cluster TaskDB #8

Open danthegoodman1 opened 2 years ago

danthegoodman1 commented 2 years ago

Table for locks, LWT to acquire lock, heartbeat lock, table is taskdb, option to create new taskdb once end of list reached or just keep looping

Configurable consistency level on write, read, and attach

danthegoodman1 commented 2 years ago

Should use unlogged batches: https://docs.scylladb.com/getting-started/dml/#unlogged-batches

danthegoodman1 commented 2 years ago

~Potentially make micro-batches, one for each partition, and do those as separete inserts?~

No because this will still be high spread on the pkeys since we can't guess which partition each will go into