WIPACrepo / pyglidein

Some python scripts to launch HTCondor glideins
MIT License
7 stars 20 forks source link

add submitter information to classads #80

Closed dsschult closed 7 years ago

dsschult commented 7 years ago

We record the "site" we run at right now.

We should be finer grained for OSG, recording the submitting site and the running site.

dsschult commented 7 years ago

Testing out "GLIDEIN_SiteResource" on OSG, which is filled with the OSG $GLIDEIN_ResourceName.

dsschult commented 7 years ago

Note that we can't use "GLIDEIN_ResourceName" ourself, or the gratia probe will get very confused and will mess up our site classification.

dsschult commented 7 years ago

This looks good enough:

dschultz@sub-1 ~ $ condor_status -af GLIDEIN_SiteResource|grep -v undefined|grep -v none|sort|uniq -c|sort -rn
    251 BNL-ATLAS
    190 MWT2
    167 IIT_CE1
    123 UConn-OSG
    118 FIU_HPCOSG_CE
    115 Hyak
     90 NUMEP_CE
     64 CIT_CMS_T2
     63 NWICG_NDCMS
     49 UCD
     46 cinvestav
     40 UCR-HEP
     26 IIT_CE2
     24 GLOW
     20 AGLT2
     13 OSG_US_USD_LEGACY
     13 BU_ATLAS_Tier2
     12 SPRACE
     10 SWT2_CPB
      9 uprm-cms
      3 OSG_US_WSU_GRID
      2 USCMS-FNAL-WC1
      2 SMU_ManeFrame_CE
      2 Sandhills
gonzalomerino commented 7 years ago

Hi,

It seems that we fill in this attribute only for the pyglidein sites osgconnect and xsede-osg that send glideins to may other OSG sites.

sub-1 ~ $ condor_history -limit 100 -af MATCH_EXP_JOBGLIDEIN_ResourceName MachineAttrGLIDEIN_SiteResource0 | sort | uniq -c 2 CHTC undefined 2 DESY undefined 2 LIDO_Dortmund undefined 1 NPX undefined 1 osgconnect BNL-ATLAS 1 osgconnect IIT_CE1 4 osgconnect MWT2 2 osgconnect none 54 osgconnect NWICG_NDCMS 1 osgconnect UCR-HEP 6 parallel undefined 4 T2B_BE_IIHE undefined 8 xsede-osg Hyak 1 xsede-osg IIT_CE1 2 xsede-osg IIT_CE2 3 xsede-osg MWT2 2 xsede-osg NWICG_NDCMS 2 xsede-osg UConn-OSG 2 xsede-osg UCR-HEP

Wouldn't it be good to always fill in the GLIDEIN_SiteResource so that it will always contain the site where the job has run? I think this will make it easier if we want to make plots or do analysis in ES/grafana.