search

dmwm / PHEDEX

CMS data-placement suite
8 stars 18 forks source link

FileDownload FTS3 backend: alert for CANCELED files #1061

Closed nikmagini closed 7 years ago

nikmagini commented 7 years ago

Seen occasionally running tranfers with FTS3 backend:

2016-10-21 08:44:13: QMon[3183]: alert: Unknown file-state: CANCELED

While we don't cancel transfers ourselves, it seems that FTS3 is now setting state to 'CANCELED' for files hitting the transfer marker timeout:

Source: srm://stormfe1.pi.infn.it:8444/srm/managerv2?SFN=/cms/store/data/Run2016G/DoubleEG/MINIAOD/23Sep2016-v1/50000/3C97FBDD-CD88-E611-A588-20CF3027A5EF.root Destination: gsiftp://eoscmsftp.cern.ch//eos/cms/store/data/Run2016G/DoubleEG/MINIAOD/23Sep2016-v1/50000/3C97FBDD-CD88-E611-A588-20CF3027A5EF.root State: CANCELED Reason: TRANSFER TRANSFER Transfer canceled because the gsiftp performance marker timeout of 360 seconds has been exceeded, or all performance markers during that period indicated zero bytes transferred Duration: 1370 Staging: 0 Retries: 0

Not a critical issue since PhEDEx will eventually expire the transfer task and resubmit it; but it will make the resubmission slower.

We forgot to add the 'CANCELED' state for files because it was not listed explicitly in the FTS3 documentation...

nikmagini commented 7 years ago

Fix merged, we'll need a 4.2.1 to distribute the fix.

nikmagini commented 7 years ago

4.2.1 released on Nov 4th