dmwm / PHEDEX

CMS data-placement suite
8 stars 18 forks source link

FileDownload FTS3 backend does not respect submission limits #1063

Closed nikmagini closed 7 years ago

nikmagini commented 7 years ago

Observed in transfers to T3_US_HEPCloud with PhEDEx 4.2.0 FTS3 backend: the agent submitted all ~1000 transfers in its queue to FTS3, resulting in a queue of ~900 pending files in FTS3 despite the default link pending limit of 50 in the agent config.

Reason: the check in the 'isBusy' function has not been updated to take into account the new pending states in FTS3:

https://github.com/dmwm/PHEDEX/blob/master/perl_lib/PHEDEX/Transfer/FTS3.pm#L258

In particular, "SUBMITTED" - so the agent treats all files as running, and submits more...

nikmagini commented 7 years ago

4.2.1 released on Nov 4th