patch to dataset ordering compareDS (miniaod)

franzoni commented 10 years ago

this function compareDS needs scratching and re-writing
this PR is intended as a possible pro-tempore solution to assure miniaod are NOT on top of AODSIM,RECO when a single request
NOTE: requires processing string to be the same for all data tiers, or, in case of ALCARECO, that processing be a longer string than normal processing string

vlimant commented 10 years ago

In relation to this and an failed attempt at getting this function rationalized, I came to the conclusion that there was no "good" way to present any of the produced dataset (even more with TaskChain involving trees of requests) and that stats should go toward displaying all outputs (with corresponding expected). On McM side, https://github.com/vlimant/cmsPdmV/commit/eef36f9d28c221e0b9e293bb3d8c28e28cd1156d as part of https://github.com/cms-PdmV/cmsPdmV/pull/665 offers a way of using the last (sequence) first (tier) for accounting, instead of the one presented by stats, and having the list of output ordered along the sequences of the request.

anorkus commented 10 years ago

Hi,

for now tested only on pdmserv* serquests with more than 2 output DS. Got a miss match (compared to existing request data in statsDB) for 1 request out of 45. pdmvserv_task_SUS-Summer12WMLHE-00004__v1_T_141003_120119_9755 produced: ['/DYJetsToLL_M-10to50_HT-400toInf_TuneZ2star_8TeV-madgraph-tauola_TaskChainWithEff/Summer12-START53_V7C-v1/GEN-SIM', '/DYJetsToLL_M-10to50_HT-400toInf_TuneZ2star_8TeV-madgraph-tauola_TaskChainWithEff/Summer12WMLHE-START53_V7C-v1/GEN', '/DYJetsToLL_M-10to50_HT-400toInf_TuneZ2star_8TeV-madgraph-tauola_TaskChainWithEff/Summer12DR53X-PU_S10_START53_V19-v1/AODSIM', '/DYJetsToLL_M-10to50_HT-400toInf_TuneZ2star_8TeV-madgraph-tauola_TaskChainWithEff/Summer12DR53X-PU_S10_START53_V19-v1/DQM', '/DYJetsToLL_M-10to50_HT-400toInf_TuneZ2star_8TeV-madgraph-tauola_TaskChainWithEff/Summer12DR53X-PU_S10_START53_V19-v1/GEN-SIM-RAW']

in this case the request output_dataset would be set for "GEN-SIM".

While in DB it has: ["///AODSIM", "///DQM", "///GEN-SIM-RAW", "///GEN-SIM", "///GEN"] and output_ds is "AODSIM"

cms-PdmV / stats

patch to dataset ordering compareDS (miniaod) #18