search

amark / gun

An open source cybersecurity protocol for syncing decentralized graph data.
https://gun.eco/docs
Other
18.08k stars 1.16k forks source link

Question: How to split and distribute work? #1368

Closed spxneo closed 6 months ago

spxneo commented 6 months ago

I'm imagining a scenario where workers received distributed work, a large chunk is split up so as to not overload 1 worker. More workers can join and leave as they please but they also need to write the data somewhere.

I was thinking of a queue system because I the order of the work done is very important. It could be short lived and long lived (might take a week)

Is this all possible to implement with gun? asking here because not sure where else to

amark commented 6 months ago

@spxneo because GUN is at-least-once delivery with strong-EVENTUAL-consistent guarantees, it isn't best (by itself, out of the box) suited for exactly-once work queues UNLESS you write some app code to use GUN to check/manage for reducing/deduplicating work ( https://gun.eco/docs/CAP-Theorem ). Would love to see a module like that tho!