search

nptscot / npt

Data processing code, also use this repo for issue tracking for the Network Planning Tool. See https://nptscot.github.io for development version
https://www.npt.scot/
GNU Affero General Public License v3.0
5 stars 0 forks source link

Batch routing pipeline seems to submit duplicate jobs #430

Closed mvl22 closed 3 months ago

mvl22 commented 3 months ago

We are seeing a fair number of cases where seemingly identical batch route jobs are being created, and succeeding without errors. This obviously means a lot of wasted CPU time which presumably slows down the whole build time. Sometimes we are seeing the same thing even 3 or 4 times.

Possibly this points to some kind of bug in the pipeline, perhaps in asyncronous logic if you are using that. Or maybe if you are doing a completion check some kind of floating point comparison issue?

Obviously if this is just different developers doing independent runs, fair enough, but it feels more like this is a bug as these jobs get reasonably clumped together.

Index screen - note 8837 (created at 2024-03-17 06:51:46), 8838 (created at 2024-03-17 07:30:49) have the same parameters:

Screenshot 2024-03-23 at 14 00 34

Job 8837 (created at 2024-03-17 06:51:46) - note that it has the same geometry as 8838 below:

Screenshot 2024-03-23 at 14 00 45

Job 8838 (created at 2024-03-17 07:30:49):

Screenshot 2024-03-23 at 14 00 49

Robinlovelace commented 3 months ago

This was a temporary issue due to multiple instances of the data workflow automation system {targets} working at the same time I think. Closing for now, will test and re-open if it emerges again.

Did notice it, thanks for flagging.