Accessing the Count property of the underlying collection

[mcavanagh]

Apologies if I'm misusing the AsyncProducerConsumerQueue<T>, but I'd like to be able to to access the Count property of the underlying Queue<T> so that I can tell when all my async queues are empty. Is this deliberately not surfaced in the API?

[IanYates]

I'd say you're right, since the Count property might be 0 when you check it, but then, before your next line of code executes, another entry could have entered the queue. Is your empty check a desire to see when everything's finished completely or more just to see if there's extra work to do right now? IIn either case, use DequeueAsync (or its Try-variant if you're just trying rather than waiting) and flag your completion of adding using CompleteAddingAsync. Once a queue is flagged as completed adding and is then completely drained you'll find the Dequeue operations will fail (see the XML comments on the methods in the wiki).

[StephenCleary]

Ian's right. I tend not to surface details if there's a high potential for misuse; Count in this case is only approximate. The appropriate way to signal completion of a queue is to call CompleteAdding and then your consumers can detect when it's complete via OutputAvailableAsync (returns false), GetConsumingEnumerable (the enumerable ends), TryDequeueAsync/TryDequeue (Success is false), or DequeueAsync/Dequeue (throws InvalidOperationException).

If this isn't sufficient, and you really do need Count for some other reason, just reopen this issue with a comment describing the use case.

[mcavanagh]

Thanks for clarifying. I suspected I was looking at it the wrong way. Like you say, there's concurrency issues around atomically checking the value of Count. My use case is that I have a few AsyncProducerConsumerQueues containing URIs which are fetched by a limited pool of workers. The workers might then enqueue more items to one or more queues. In order for my program to finish, I check whether there are any HTTP requests in progress, and whether all of the queues are empty (hence wanting to check Count). I probably need to rethink the approach, but I don't want to just 'wait for an arbitrary amount of time' before calling CompleteAdding on them.

[StephenCleary]

Hmmm, the PCQs were not designed with this in mind - the same code being producer and consumer. Though I suppose it would work. (Side note: TPL Dataflow was designed with this in mind).

It seems to me the most straightforward approach would be to make your "in progress" counter an "unfinised" counter instead - increment it when inserting into the queues and decrement it only when the HTTP request is fully processed. Would that work for you?

[mcavanagh]

Yeah, something along those lines. I was investigating TPL Dataflow when I found a StackOverflow post where you linked AsyncEx. I think I can make this work using your library, but I'll keep Dataflow in mind. Thanks.