concurrentqueue and readerwriterqueue,which one has better performance?

[libaineu2004]

If the concurrentqueue project uses a single producer and a single consumer, compared with the readerwriterqueue project, which one has better performance?

[cameron314]

ReaderWriterQueue will generally be faster. I encourage you to benchmark your own specific use case.

[libaineu2004]

ReaderWriterQueue will generally be faster. I encourage you to benchmark your own specific use case.

Then I have another question, what is the purpose of Tokens? Faster or lower memory footprint? How much performance improvement?

[cameron314]

Tokens improve throughput. My benchmark results are published.

[libaineu2004]

Tokens improve throughput. My benchmark results are published.

Can you briefly describe how it works? Why does it improve performance?

[cameron314]

From the README:

There's usually two versions of each method, one "explicit" version that takes a user-allocated per-producer or per-consumer token, and one "implicit" version that works without tokens. Using the explicit methods is almost always faster (though not necessarily by a huge factor). Apart from performance, the primary distinction between them is their sub-queue allocation behaviour for enqueue operations: Using the implicit enqueue methods causes an automatically-allocated thread-local producer sub-queue to be allocated. Explicit producers, on the other hand, are tied directly to their tokens' lifetimes (but are recycled internally).

In order to avoid the number of sub-queues growing without bound, implicit producers are marked for reuse once their thread exits. However, this is not supported on all platforms. If using the queue from short-lived threads, it is recommended to use explicit producer tokens instead.

See also https://github.com/cameron314/concurrentqueue#tokens

Briefly: Using producer tokens skips a lookup to the current thread's implicit producer sub-queue, and using consumer tokens spreads the load better across all sub-queues.