search

dmwm / PHEDEX

CMS data-placement suite
8 stars 18 forks source link

Add support for moves to T1 Disk endpoints #958

Closed nikmagini closed 9 years ago

nikmagini commented 10 years ago

Feature request from Oli for CompOps:

Evaluate the possibility to add support in PhEDEx also for moves TO T1 Disk endpoints

Use-case: generate files at T2s, move to T1 Disk for further processing, then copy/move input and output to T1 MSS for custodial storage.

Need to handle with care the case in which the same dataset is moved simultaneously to T1 Disk and T1 MSS to avoid triggering unwanted deletions.

hufnagel commented 10 years ago

IMO, you cannot have move and copy requests active in the system at the same time. That will always lead to indeterminate behavior. We could have move requests to multiple locations, but cannot create or even allow copy requests until the moves are no longer active.

Or do I see this wrong given how move requests are currently defined (move data from anywhere to a single target, deleting copies anywhere else) ?

nikmagini commented 10 years ago

Hi Dirk,

for the current behavior of Move requests see here:

https://twiki.cern.ch/twiki/bin/view/CMS/PhedexProjSubscriptions#Move_subscriptions_in_4_0_0

We can summarize it as:

"Move data to one or more MSS targets, deleting all unsubscribed Disk copies anywhere else"

Note in particular "unsubscribed". This means that it's safe to create additional Move or Copy requests while a Move is already in progress, because the new destination replicas created will not be targeted for deletion by the first Move request.

So I think that it should be safe to change the feature like this:

"Move data to one or more T0/T1 Disk or MSS targets, deleting all unsubscribed Disk copies anywhere else"

Only one thing to consider: currently, when a Move request is created, PhEDEx sends a request to the source sites asking if they want to unsubscribe the data in order to trigger the deletion. We definitely don't want T1 Disk admins to trigger a deletion by mistake when a T1 MSS Move is requested. So maybe we should just deprecate this feature (assuming that a source site wants to keep the data if they already have a subscription should be a safe assumption).

N.

nikmagini commented 9 years ago

Deployed to cmsweb in April 8th release, closing.

(side note: the behaviour of Move requests at source sites was not changed in the end, leaving for a possible future enhancement)