search

bcgov / nr-rfc-processing

River forecast centre data processing solution
Apache License 2.0
4 stars 2 forks source link

Limit the number of jobs that can be processed in parrallel #100

Closed franTarkenton closed 3 months ago

franTarkenton commented 3 months ago

Sometimes the job may become deactivated, resulting in a backlog of jobs that need to be processed. In this situation you could wind up with more than a 100 watersheds that need to be processed. If one of the jobs fails in the matrix, then all of the jobs will fail. For this reason, combined with not wanting to use up all the runners avaible to bcgov org, we want to limit the job to run 10 jobs at a time max.

franTarkenton commented 3 months ago

This has been completed... added the max parallel clause to the github action that implements the matrix processing