Open siennathesane opened 7 months ago
I guess you want a variant of barrier that does not need to know the number of threads at construction, right? https://docs.rs/crossbeam-utils/latest/crossbeam_utils/sync/struct.WaitGroup.html#wait-groups-vs-barriers
I guess you want a variant of barrier that does not need to know the number of threads at construction, right?
And is also potentially scalable, as well. WaitGroup
allows users to add or remove threads as needed, which makes it useful as a "dumb barrier" of sorts. I can likely use a barrier as a drop-in replacement for now (haven't tried yet), but I wanted to surface this type of use case.
I have a complex use case for
crossbeam_utils::sync::WaitGroup
. I have a thread-safe structure that builds SSTables for databases, and it is very large in-memory, so it can't easily be cloned. As part of that, the table builder needs to be able to compress and encrypt the blocks when they're complete, but that work is done asynchronously. I want to useWaitGroup
to provide the final sync point before I flush the SSTable to disk.Here's my current MVCE (playground link).
Because of the size of the data in the blocks, and the overall builder size, moving the data around isn't very feasible, so I mutate and operate on it in-place as much as possible. I pass pointers around, and everything is behind
Arc
, and most are also locked with mutexes. However, when I tried to introduceWaitGroup
, I got this error:I've tried several different attemptes with pointers, smart pointers, and mutexes, but I cannot seem to get
WaitGroup
to meet my needs. It would be great ifWaitGroup
could support this kind of use case.