search

dmwm / CRAB2

CRAB2
2 stars 11 forks source link

validate user output file name against LFN in Lexicon before submitting #1156

Closed belforte closed 9 years ago

belforte commented 9 years ago

see https://hypernews.cern.ch/HyperNews/CMS/get/crabFeedback/7673/2/1/1/1.html

belforte commented 9 years ago

@mmascher does this apply to Crab3 as well ? Or do you have this check already ? In crab2 I had a check on the directory name, not on the file name (which may not even be in crab.cfg but only in the cmssw one).

mmascher commented 9 years ago

Hi Stefano,

do you know which method of Lexicon DBS is using to validate the filename? That's the regexp we are using to validate the edmfiles: https://github.com/dmwm/CRABServer/blob/master/src/python/CRABInterface/Regexps.py#L35 , which contains a '.'... (In addition I am checking that it's less than 255 chars)

If there is a single regexp I can extract from Lexicon, like I did for the searchdataset (https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Lexicon.py#L78) I can import it and use it. Maybe it's https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Lexicon.py#L31 ?

belforte commented 9 years ago

need to look it up, it is tricky because LFN for CRAB is more free then for WMA. It will be something like Lexicon.lfn(name) https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Lexicon.py#L240 But there is also https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Lexicon.py#L352

as usual it seemed all clear at the time I was using this for publication (last winter), then time passed ... :-(

mmascher commented 9 years ago

I was fearing it was the lfn method. I am not happy about calling it in the CRAB server, it does check 10 different regexps. I prefer to extract the single regexps we need and use them, as I did for the serachdataset.

Maybe @yuyiguo can suggest me where to look in DBS3 for the regexps it uses

yuyiguo commented 9 years ago

https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Lexicon.py#L240 is what used in DBS. I don't know who is using the userlfn. There are morn than one type of lfns used in CMS and DBS has to check it in a single function call.

belforte commented 9 years ago

@mmascher that's also what Crab2 does. You should stay with that, or suggest a modifciation to the Lexicon, but not build your own version of it, less things diverge in long run.

belforte commented 9 years ago

Given that we never did this in Crab2, and only had one problem with one user once, and that Crab2 is end of life. And given that messing with the code that extracts output file names from config. is dangerous. Let's live with this until Crab3.