search

dmwm / CRABClient

runrange
14 stars 36 forks source link

Bug in crab report --recovery options #4736

Closed lecriste closed 1 month ago

lecriste commented 6 years ago
belforte commented 6 years ago

I ran the client with pdb and figured out that the problem is server side, From line 111 of report.py lumisToProcessPerJob is a list of empty dictionaries. So the client does not know what was supposed to be processed and can't creare the list of what is to be recovered. Refs: https://github.com/dmwm/CRABClient/blob/master/src/python/CRABClient/Commands/report.py#L111 https://github.com/dmwm/CRABClient/blob/master/src/python/CRABClient/Commands/report.py#L281-L333

belforte commented 6 years ago

we should anyhow keep this open until server side has been understood, possibly fixed and result verified in crab report

belforte commented 6 years ago

Nopw. List of lumis to be processed should be retrieved from webdir. More investigation is needed.

belforte commented 6 years ago

I was testing the code on this task which was reported y a user last week: https://cmsweb.cern.ch/crabserver/ui/task/180121_124727%3Agsfs_crab_EPOS_MinBias_RECODEBUG_XeXe_9_4_Full

where the recovery lumimasks are missing as in the initial issue comment. Eventually I found that this is because the task used config.Data.splitting = 'FileBased' and in this case the run_and_lumis.tar.gz has no info on the input lumi list for each file Basically recovery=notPublished works only with splitting by lumi. Of course client could be modified to generare the recovery mask using the list of lumis in each file from DBS etc. I vaguely remember a discussion on this with past developers (Andres ? Emilis ?). Anyhow I think the best is to document the situation. Users can use lumimask algebra starting from DBS as already documented to create a lumimaks in this case.

@lecriste plese make sure that documentation and crab report --help and output report this fact.

belforte commented 9 months ago

@mapellidario can you easily and quickly check if this is overridden by new crab recovery and close ? If it takes more than 30min, let's close anyhow. It is 5 years old and I have not heard anybody complaining

belforte commented 1 month ago

all fixed now !