Closed andrew-regan closed 7 years ago
Capturing some comments from Slack related to this from @m4dcoder:
"in current implementation, there's a soft FIFO queue. on completion of the last action instance, if there's any delayed execution, st2 policy engine will look for the earliest delayed from the DB. but from the log entries, here's what i can deduce. there's already an instance of the action running at 2017-01-17 07:49:31 so 587dcc8bb4eaa909d5ce0ebf was put on delay. at 2017-01-17 07:49:41,413, the previous action execution already finished and while the policy engine is still getting ready, 587dcc94b4eaa909d8bcc212 was in the queue and scheduled. at 2017-01-17 07:49:41,708, 587dcc8bb4eaa909d5ce0ebf was evaluated again and it was cut in line by milliseconds..."
"there's a separate queue for actions being requested and those that are delayed atm."
Any update on this? @bigmstone Any chance the "to be verified" label means there's a fix in the works?
@andrew-regan I've not verified this bug and am on another issue at the moment. @lakshmi-kannan is this something you could verify this week while you're on support?
@bigmstone is the fix in a dev branch somewhere? Didn't see a PR. Thanks!
@andrew-regan I think that what he is saying is that "to be verified" means someone needs to check through this and verify that yes, this is indeed a bug, it is related to those initial thoughts from Winson, and is reproducible.
It doesn't mean "there is a fix that needs to be tested and here is some code." If there was a PR, it would be tagged with this issue, and the comments would read something like "here is a potential fix for this problem, it needs to be tested to make sure it does resolve this issue"
Gotcha...just wasn't sure what the label implied. I'm upgrading to 2.2. Will let you know if it reproduces there as well.
@andrew-regan and update on if you could reproduce in 2.2?
Still waiting on some test results from QA. We're at 2.2.1 so will let you know if we see it there.
@andrew-regan thanks. I'm tracking this issue, so I'll help you get it closed.
Just an update. Testing with StackStorm 2.2.1 is starting on our side. Will let you know if we see this reproduce again.
@bigmstone, sorry for the delay...we are no longer seeing this issue with 2.2.1.
Thanks for the update @andrew-regan. Let's close it for now, re-open if it occurs in a current release
Still seeing the same thing. The workflow is put into delay by concurrency policy and the order of execution is newest to oldest. Why the first queued workflow is running later on? Stackstorm version being used is: st2 3.5.0, on Python 3.6.8
@zsmanjot This issue is from 2017 and marked as closed after confirmation that it was no longer a problem in version 2.2.1. Please create a new issue with any relevant logs and details to the issue you're seeing as the code base has evolved considerably over the last 4.5 years.
Sure @nzlosh I am opening a new issue for this.
I have an Action with a concurrency policy that only allows 1 instance of the Action to run at a time. We’re intermittently seeing cases where multiple triggers are dispatched that lead to this Action being invoked close to each other. Strangely the Actions are not being invoked in the order I’d expect. I was under the impression that there’s a FIFO queue that controls how Actions are dispatched to runners that would cause the Actions to run in order? The snippet bellow shows the logs for 2 executions of this Action. In the snippet you can see that the logs indicate 587dcc8bb4eaa909d5ce0ebf was requested first, but it eventually gets delayed and 587dcc94b4eaa909d8bcc212 gets dispatched to a runner first.
This is seen in 2.1.1.