Optimising Bulk Enqueue for memory constrained environments.

[BlamKiwi]

We're using the queue in an environment where we can't dynamically allocate more memory to the queue after initial creation. (We additionally only allow explicit consumers/producers)

Unlike try_dequeue_bulk, try_enqueue_bulk is an all or nothing interface. We have a code-path that tries to do a bulk enqueue before resorting to enqueing items individually.

After some testing it looks like it's never succeeding in practice. It looks like using the queue in this way means blocks never get returned to the freelist, since a block is always associated with a producer token.

What would be the appropriate fix for this? Would it be as simple as over-specifying the number of producers at construction?

[BlamKiwi]

Don't worry, for some reason increasing the block size fixed this issue.

[cameron314]

If you're only using try_enqueue_*, you're probably running out of block index memory, even if you have enough blocks available. In addition to increasing BLOCK_SIZE, try increasing EXPLICIT_INITIAL_INDEX_SIZE