default black list does not work for remoteGlidein

[ericvaandering]

Original Savannah ticket 101186 reported by belforte on Fri Apr 12 16:43:07 2013.

when I implemented "SE only" for remoteGlidein, I decided to ignore the default black list, since current crab code only converts it to a list of CE's. I thought "if a site is bad, we do not send pilots". But now I think it was a mistake. We need the possibility to black list sites also in crab2, non only in factory, because there are cases other then "bad site" where the user must be able to decide to override the default black list. E.g.https://savannah.cern.ch/support/index.php?136688

[ericvaandering]

Comment by belforte on Tue Apr 23 04:42:00 2013

committed, se black list is applied initially at data location discovery time: /local/reps/CMSSW/COMP/CRAB/python/Splitter.py,v <-- Splitter.py new revision: 1.57; previous revision: 1.56

[ericvaandering]

Comment by belforte on Fri May 3 14:10:05 2013

previous commit does not work if there is no SE_black_list in crab.cfg as crab -creates prints

crab: SE black list applied to data location: [',', 'T', '1', '', 'D', 'E', ',', 'T', '1', '', 'E', 'S', ',', 'T', '1', '', 'F', 'R', ',', 'T', '1', '', 'I', 'T', ',', 'T', '1', '', 'T', 'W', ',', 'T', '1', '', 'U', 'K', ',', 'T', '1', '', 'U', 'S', ',', 'T', '3', '', 'U', 'S', '', 'U', 'C', 'L', 'A', ',', 'T', '3', '', 'U', 'S', '', 'V', 'a', 'n', 'd', 'e', 'r', 'b', 'i', 'l', 't', '', 'E', 'C', '2', '\n']

see https://hypernews.cern.ch/HyperNews/CMS/get/crabFeedback/6701/1.html

[ericvaandering]

Comment by belforte on Sat May 4 03:27:21 2013

committed a fix /local/reps/CMSSW/COMP/CRAB/python/Splitter.py,v <-- Splitter.py new revision: 1.58; previous revision: 1.57 and created CRAB_2_8_7_patch1

but still problems: https://hypernews.cern.ch/HyperNews/CMS/get/crabFeedback/6702.html i.e. File "/afs/cern.ch/cms/ccs/wm/scripts/Crab/CRAB_2_8_7_patch1/python/Splitter.py", line 50, in init seBlackList += blackAnaOps.strip().split(',') TypeError: cannot concatenate 'str' and 'list' objects

[ericvaandering]

Comment by belforte on Sat May 4 03:31:59 2013

fixed again. Problem was original sin in Splitter.py by original developers to intialize seBlackList to empty list [] instead of empty string "" in case it is not present in crab.cfg. Let's make sure which use case I verify:

1. minimal crab.cfg, all defaults
2. add se_black_list to crab.cfg
3. add remove_default_blacklist=1 to crab.cfg
4. do 2. and 3. at same time

do the same four for remoteGlidein and glite

[ericvaandering]

Comment by belforte on Sat May 4 04:09:55 2013

also do 1., 2., 3. and 4. with an se_white_list in addition

[ericvaandering]

Comment by belforte on Sat May 4 05:37:17 2013

ok now I make sure that the se b/w list is always a list [] after initial filling and it seems OK. Verified 1. 2. 3. 4. both with and without an additional white list both for remoteGlidein and glite /local/reps/CMSSW/COMP/CRAB/python/Splitter.py,v <-- Splitter.py new revision: 1.59; previous revision: 1.58

[ericvaandering]

Closed by belforte on Fri May 10 09:08:38 2013

[ericvaandering]

Comment by belforte on Fri May 10 09:08:38 2013

releases in CRAB_2_8_7_patch2