search

dmwm / PHEDEX

CMS data-placement suite
8 stars 18 forks source link

Move Subscription not removing files from Disk #931

Closed ericvaandering closed 10 years ago

ericvaandering commented 10 years ago

Original Savannah ticket 101405 reported by wildish on Mon May 6 03:51:45 2013.

Dear Transfer Team,

Now that the Disk/Tape separation is done at T1_UK_RAL. We ran some requests at T1_ES_PIC and T1_UK_RAL and we subscribed (move) custodial to T1_ES_PIC_MSS:

https://cmsweb.cern.ch/phedex/datasvc/xml/prod/requestlist?dataset=/monoLeptonToLL_mDM300_V_8TeV/Summer12-START53_V7C-v1/GEN-SIM

But we find out that the files were not removed from T1_UK_RAL_Disk:

https://cmsweb.cern.ch/phedex/datasvc/xml/prod/blockreplicas?dataset=/monoLeptonToLL_mDM300_V_8TeV/Summer12-START53_V7C-v1/GEN-SIM

Any idea on what went wrong? Is it a bug?

Thank You,

Edgar

ericvaandering commented 10 years ago

Comment by magini on Mon May 6 03:39:56 2013

Hi Edgar,

it was a feature, now it is a bug :)

The BlockDelete central agent explicitly ignores all T0|T1* nodes when creating deletions for Move subscriptions, to protect against accidental deletion of custodial replicas.

This protection was OK in the past when we only had T1Buffer/MSS nodes, but now it is too restrictive - the agent should allow the creation of move deletions for T1Disk nodes.

For now you will have to request the deletion of the replicas on T1_Disk manually, after verifying that the transfer to T1MSS is complete.

Reassigning to PhEDEx bug tracker.

Cheers, Nicolo'

ericvaandering commented 10 years ago

Comment by magini on Mon May 6 03:51:45 2013

This item has been reassigned from the project CMS Computing Infrastructure Support support tracker to your tracker.

The original report is still available at support #137317

Following are the information included in the original report:

Now that the Disk/Tape separation is done at T1_UK_RAL. We ran some requests at T1_ES_PIC and T1_UK_RAL and we subscribed (move) custodial to T1_ES_PIC_MSS:

https://cmsweb.cern.ch/phedex/datasvc/xml/prod/requestlist?dataset=/monoLeptonToLL_mDM300_V_8TeV/Summer12-START53_V7C-v1/GEN-SIM

But we find out that the files were not removed from T1_UK_RAL_Disk:

https://cmsweb.cern.ch/phedex/datasvc/xml/prod/blockreplicas?dataset=/monoLeptonToLL_mDM300_V_8TeV/Summer12-START53_V7C-v1/GEN-SIM

Any idea on what went wrong? Is it a bug?

Thank You,

Edgar

ericvaandering commented 10 years ago

Comment by magini on Mon May 6 03:58:09 2013

Hi Tony,

two changes needed:

1) for the web/datasvc: change the following filter in PHEDEX::RequestAllocator::Core to exclude only T0/T1_Buffer/MSS nodes, allowing T1_Disk nodes:

+verbatim+

Exclude T0/T1 nodes from list of source nodes that will receive deletion request for move

    delete @sources{grep /^(T1|T0)/, keys %sources};

-verbatim-

2) for the BlockDelete agent: change the node filter in the "merge into t_dps_block_delete" statement in the same way, excluding only T0/T1_Buffer/MSS nodes, allowing T1_Disk nodes.

Cheers, Nicolo'

ericvaandering commented 10 years ago

Comment by wildish on Thu Aug 22 02:59:53 2013

This is fixed in 4.1.3, which will be released shortly

ericvaandering commented 10 years ago

Closed by wildish on Thu Aug 22 02:59:53 2013