search

dmwm / CRAB2

CRAB2
2 stars 11 forks source link

enable overriding data location for global data access (AAA) #816

Closed ericvaandering closed 10 years ago

ericvaandering commented 10 years ago

Original Savannah ticket 97801 reported by belforte on Fri Sep 28 02:34:15 2012.

see https://hypernews.cern.ch/HyperNews/CMS/get/wanaccess/9/2/1/1/1.html

ericvaandering commented 10 years ago

Comment by gowdy on Fri Sep 28 07:04:13 2012

Should we not put the global redirector in, instead of the US one? It is cms-xrd-global.cern.ch.

ericvaandering commented 10 years ago

Comment by belforte on Fri Sep 28 07:24:25 2012

that as well, let's make sure we define what we want first, then worry about the code. AFAIK there are three use cases:

  1. force execution site "no matter what" (which may include a way to disable glidein overlow)
  2. allow testing of the fully delocalised scenario (run anywhere, read from anywhere) w/o having every job run like that
  3. execute jobs from any user in the delocalized scenario

The initial thread was about 1. I suspect what Andrew wrote to be 3. albeit mitigated by restricting to a pool of sites.

ericvaandering commented 10 years ago

Comment by belforte on Wed Oct 24 05:06:06 2012

I think I will implement an option in crab.cfg [GRID] force_execution_site=<sitelist>

where <sitelist> has same syntax as white/black lists i.e. a comma separated list of site names possibly in compact form like T2, T3_US etc.

this will override se/ce selection

I will do for remoteGlidein first, then look at other schedulers

ericvaandering commented 10 years ago

Comment by belforte on Fri Oct 26 14:32:25 2012

all in all [CMSSW] data_location_override=<site> seems better will simply replace whatever DBS returns with this and everything else stays the same. That will be in cms_cmssw.py where blockSites is filled.

ericvaandering commented 10 years ago

Comment by belforte on Fri Oct 26 15:45:34 2012

in the end decided to put new paramenter in [GRID] section for similarity with SE/CE B/W list, so [GRID] data_location_override = T2_IT_Bari,T2_UK etc.

/local/reps/CMSSW/COMP/CRAB/python/cms_cmssw.py,v <-- cms_cmssw.py new revision: 1.388; previous revision: 1.387

ericvaandering commented 10 years ago

Closed by belforte on Thu Dec 20 08:01:24 2012

ericvaandering commented 10 years ago

Comment by belforte on Thu Dec 20 08:01:24 2012

released in CRAB 2_8_4