Fix WMSpec.WMSteps.Executors Unittests

[DMWMBot]

Basically just make them integration - buildbot is not responding well to StageOut requests.

[sfoulkes]

sfoulkes: ======================================================================

ERROR: testOnWorkerNodes (main.otherStageOutTest)

Traceback (most recent call last): File "StageOut_t.py", line 278, in testOnWorkerNodes raise RuntimeError RuntimeError

FAIL: testCPBackendStageOutAgainstReportNew (main.otherStageOutTest)

Traceback (most recent call last): File "StageOutt.py", line 227, in testCPBackendStageOutAgainstReportNew executor.execute( ) File "/data/sfoulkes/WMCORE/src/python/WMCore/WMSpec/Steps/Executors/StageOut.py", line 189, in execute lfnRegEx(lfn) File "/data/sfoulkes/WMCORE/src/python/WMCore/Lexicon.py", line 93, in lfn return check(regexp3, candidate) File "/data/sfoulkes/WMCORE/src/python/WMCore/Lexicon.py", line 124, in check "'%s' does not match regular expression %s" % (candidate, regexp) AssertionError: 'hosts' does not match regular expression /store/(temp/)*(user|group)/([a-zA-Z0-9]+)/([a-zA-Z0-9-]+)/([a-zA-Z0-9-]+)/([a-zA-Z0-9-]+)/(09]+)/([a-zA-Z0-9-_]+).root

FAIL: testCPBackendStageOutAgainstReportOld (main.otherStageOutTest)

Traceback (most recent call last): File "StageOutt.py", line 257, in testCPBackendStageOutAgainstReportOld executor.execute( ) File "/data/sfoulkes/WMCORE/src/python/WMCore/WMSpec/Steps/Executors/StageOut.py", line 189, in execute lfnRegEx(lfn) File "/data/sfoulkes/WMCORE/src/python/WMCore/Lexicon.py", line 93, in lfn return check(regexp3, candidate) File "/data/sfoulkes/WMCORE/src/python/WMCore/Lexicon.py", line 124, in check "'%s' does not match regular expression %s" % (candidate, regexp) AssertionError: 'hosts' does not match regular expression /store/(temp/)*(user|group)/([a-zA-Z0-9]+)/([a-zA-Z0-9-]+)/([a-zA-Z0-9-]+)/([a-zA-Z0-9-]+)/(09]+)/([a-zA-Z0-9-_]+).root

---

Ran 5 tests in 10.831s

FAILED (failures=2, errors=1)

[DMWMBot]

mnorman: Tempted to change this one to "Won't fix"

Why do we have to support point-to-point on-node stage out mechanisms? How are we actually supposed to support them? Can we get some input from someone who actually knows how this is supposed to work?

[PerilousApricot]

meloam: The tests aren't for inter-node communication.

testOnWorkerNodes should be flagged with an attribute that keeps it from running except in special cases (I have a buildslave that submits the test harness via condor_g to different sites). (I don't have eclipse open, so I can't check for the attribute)

The other two tests testCPBackendStageOutAgainstReport[Old|New] do exactly what they say and test the cp backend for stageout. When I put it in, the code coverage went up a bit (it's been a bit, but IIRC, the test backends did something strange that caused them to skip some code). It appears that the new LFN-verification stuff is causing those tests to fail because I'm writing to a totally bogus location. Feel free to change the target LFN to something that makes sense.

[DMWMBot]

mnorman: What should I change? I don't quite see how you're entering stuff in the first place, or how you specify the final lfn. This confuses me because it should just spit things out earlier.

I don't know if we want to try and replace it with a rational lfn that's seven or eight sections long, or whatever it is. Will that even work?

[PerilousApricot]

meloam: I'm assigning to me. I know how to fix it, but I'm under a pretty hard deadline to get some stuff done for science (need to present results in geneva next week).

Long story short, the MergeSuccess.pkl is a pickled workflow that has the file information that gets extracted.

[DMWMBot]

mnorman: Okay,

I think the timeline horizon for this is ~infinity, so take your time.

[DMWMBot]

mnorman: Assigning this to Melo, because I can, to be finished whenever he has time (or maybe never).

[PerilousApricot]

meloam: Cool. once my pat-tuple bitch duties move to non-eternal crisis mode. I'll fix this.

[DMWMBot]

mnorman: Punting indefinitely.

[DMWMBot]

mnorman: Punting indefinitely again because Metson reassigned this to August.