Workitem with high priority within workgroup is being not picked up by STP

[gunasekaranmohan]

Hi, I have 20-30 workgroups. Each workgroup will have arroud 3 to 4k workitems with different priorities. Sometime i face a problem. Few workgroup items are not being picked up even though those workitems are having high priority. Lower priority items are being picked from other workgroup. I tried to simulate the issue, but I couldn't. This issue doesn't happen always.

Have you come across similar issue?

[amibar]

Hi, What is the concurrency of the workgroups? It looks like you have created a live lock (starvation). Assume you have 2 workgroups with concurrency of 1 each: The first has only normal priority work items and the second has high and low priority work items. Since a workgroup doesn’t queue its next item to the STP queue its previous work item has completed, it may occur that high priority items is pending in the workgroup queue waiting for a low priority work item in the STP queue. The later doesn’t complete because other STP workitems with higher priority are handled by the STP. In your case there are always high priority work items to deal with so the low priority work items block their workgroup work items to work. This is what I can come up with, does it help ? Ami

[gunasekaranmohan]

Thanks a lot for your reply.

I have 30 workgroups. Each has max concurrency as 20. Overall concurrency of STP is 100. Our process submits tasks in all work groups around in 1 hr batch. Each task may be having different priority.

E.g. Workgroup A has few low priority items and high priority items.

Workgroup B and C has normal priority items. What I am see is that Workgroup B and C items are progressing. None of items in Workgroup A is running.

Workgroup A items are not starting until B and C are complete (though workgroup A has high priority item than B and C).

It seems your explanation is closer to what is happening.

Let me further check and see what I can do further. Meantime if you have any idea to solve this issue let me know.

I think, When tasks are released from each workgroups to STP, if we change the condition in WorkItemsGroup.cs, then i see the expected result most of the time. But still it it is not perfect solution.

if (_workItemsInStpQueue < _concurrency) ===> if(_workItemsExecutingInStp < _concurrency && _workItemsInStpQueue < _concurrency)