Sentinal 2ARDA/Bsync orchestration job pushed hundreds of jobs to the PBSjobqueue. This was causing significant delays in qsub job execution.
Root cause:sync path in a single qsub job for s2_ard_granule was submitted without grouping.
for ex. a single qsub was submitted to process datasets on a daily basis instead of grouping them.
Proposed solution
Group number of days and then submit sync job. This way the max number of pbs jobs submitted in a year for s2_ard_granule is reduced to 48 from 365.
Request for pull request
Sentinal 2ARDA/Bsyncorchestration job pushed hundreds of jobs to thePBSjobqueue. This was causing significant delays inqsubjob execution. Root cause:syncpath in a singleqsubjob fors2_ard_granulewas submitted without grouping. for ex. a singleqsubwas submitted to process datasets on a daily basis instead of grouping them.Proposed solution
pbsjobs submitted in a year fors2_ard_granuleis reduced to48from365.2019year.pylinterror.