belforte
commented
10 years ago more from Eric:
The answer is “no”. But preferring global over phys03 is not necessarily right either. You can have a USER dataset in phys03, migrated to global by StoreResults and then extended on phys03.
So I think the white list of Tiers that can come from phys0{1,2,3} is the best available solution.
On Jul 15, 2014, at 10:25, Stefano Belforte stefano.belforte@cern.ch wrote:
I see, so the answer is no ? Initially I was thinking of plainly check for the dataset in global scope DBS as well, and possibly silently do the right thing, but was afraid that crab3 client could not do it. It Crab3 client contacts DBS alreaedy, we can surely do the direct check. stefano
On 07/15/2014 05:22 PM, Eric W Vaandering wrote:
I wanted to see if there was a feature we could use during data discovery that would flag a RECO dataset in phys03 as having been migrated from somewhere else. In which case you could refuse to use it. In other words, not do it with set Tier names but really trying to detect the problem you are trying to fix.
Eric
On Jul 15, 2014, at 10:18, Stefano Belforte stefano.belforte@cern.ch wrote:
?? I can't undrestand what you mean here. But assume you wanted to say that datasets other then USER so far are always migrated to local scope instance, not created there.
On 07/15/2014 05:14 PM, Eric W Vaandering wrote:
I asked Yuyi if there was any indication in DBS that a dataset was migrated to an instance rather than "born" there, but there is not.
see https://hypernews.cern.ch/HyperNews/CMS/get/wmDevelopment/615/1/1/1.html i.e.
*\ Discussion title: Workflow Management Development
I'd like to propose a twist of this: we only accept in CRAB (2 or 3) a phys0x local scope DBS instance if the requeste dataset has tier=USER
Anybody sees something wrong ?
Rational: try to prevent users from screwing up early rather then late, to save our time in diagnosing. I fear that many will keep phys03 url around when not needed and access oldish versions of official datasets that were migrated a few days or weeks ago.
I was thinking of printing a BIG WARNING, but ..
If we do the proposal above, Crab3 can detect at config. parsing time in the client.
Stefano