cameron314 / concurrentqueue

A fast multi-producer, multi-consumer lock-free concurrent queue for C++11
Other
9.52k stars 1.66k forks source link

To use one concurrent queue between multiple threads #377

Open harshad-0302 opened 3 months ago

harshad-0302 commented 3 months ago

My use case is that I have one function which will enqueue data in the queue & I have multiple threads which will dequeue & use this data at the same time one by one.

Is it possible with concurrent queue by only using single queue ? The issue is if first thread dequeue's a data element from the queue then that data element will no more be available in the queue so how the other threads will dequeue same data element & use it?

So is there any way that only one dequeue operation happens between all the threads?

Or else any other way to resolve this using concurrent queue please suggest.

I have gone through sample examples but didn't find any example which fits my use case. So writing here, please provide your input.

cameron314 commented 3 months ago

This is not possible with ConcurrentQueue. It's a data structure, but you seem to be searching for something closer to a messaging system.

You could, however, use N queues for N dequeueing threads, and enqueue N copies of the same data to all of them. But obviously this is suboptimal.

harshad-0302 commented 3 months ago

I saw that concurrent queue supports multiple consumers so if we assume each thread to be a consumer . Then is there any possibility that multiple consumer(threads) consume same data from the same queue?

If not then what is the exact benefit of multiple consumers ?

cameron314 commented 3 months ago

Not possible with this data structure.

Example use: Consider a threadpool processing a series of jobs. Each thread pulls a different job from the work queue.