search

OpenFn / lightning

OpenFn/Lightning ⚡️ is the newest version of the OpenFn DPG and provides a web UI to visually manage complex workflow automation projects.
https://openfn.github.io/lightning/
GNU Lesser General Public License v3.0
115 stars 31 forks source link

Race condition when triggering sync from openfn while deploy actions are running #2179

Open midigofrank opened 3 weeks ago

midigofrank commented 3 weeks ago

Will add more details later..

@mtuchi ran into a situation where one of his deploy workflows could not complete because his branch was out of date with the remote branch. Know that git error which tells you to pull from the remote first?

He said he did a push to his branch and then triggered a sync from openfn. My gut tells me that the sync from lightning did a commit before one of his workflows could complete. I think this is possible given that the deploy workflows are currently running sequentially, while the pull ones aren't.

Maybe we should have the pull actions running in the openfn-deployment group as well so that we avoid all these race conditions.

midigofrank commented 3 weeks ago

@taylordowns2000 thoughts?

taylordowns2000 commented 3 weeks ago

Makes sense. I didn't realize that our fix didn't put all these workflows in the same group. Please raise a PR. I see no harm in forcing all the OpenFn actions to be sequential.

christad92 commented 2 weeks ago

@midigofrank did we fix this?

midigofrank commented 2 weeks ago

@christad92 nope.

midigofrank commented 1 week ago

Oops!! This isn't as straight forward as I thought https://taurit.pl/github-canceling-since-a-higher-priority-waiting-request-exists/