search

uwcms / FinalStateAnalysis

An analysis framework for the Compact Muon Solenoid
4 stars 43 forks source link

PAT tuplize VH leptonic decay signal samples #11

Closed ekfriis closed 11 years ago

ekfriis commented 11 years ago

Dear Tapas,

Can you please PAT tuplize the following samples in 53X (8TeV ones) and 4XX (7TeV ones)?

The new samples have been added in the data7TeV.py and data8TeVNew.py in this commit:

6f86c613b80c989c1fba847bac343b9dd53935c7

The samples to be processed are:

VHWW_lepdecay_110
VHWW_lepdecay_120
VHWW_lepdecay_130
VHWW_lepdecay_140
VHtautau_lepdecay_110 VHtautau_lepdecay_115 VHtautau_lepdecay_120 VHtautau_lepdecay_125 VHtautau_lepdecay_130 VHtautau_lepdecay_135 VHtautau_lepdecay_140

(you can select them by --samples "VH*lepdecay*")

Thanks!

Evan

tsarangi commented 11 years ago

8 TeV ones are submitted.

For the 7 TeV samples, should I take the recent changes done by Ian ?

ekfriis commented 11 years ago

Yes, please do. Thanks!

iross commented 11 years ago

We haven't confirmed that my proposed changes are safe... so I haven't committed anything yet.

ekfriis commented 11 years ago

I think it will be okay, and these samples will be fast.

mcepeda commented 11 years ago

I'm checking, give me 2 seconds.

tsarangi commented 11 years ago

do i need a recompilation after pulling the changes into local repo ?

mcepeda commented 11 years ago

You will need to recompile,yes.

I saw no problems, but Ian told me I should have seen them :-). Checking more carefully now.

iross commented 11 years ago

Hi Tapas-- Yes, you will.

ekfriis commented 11 years ago

Hi Tapas,

I think anytime the PAT changes I would just create a new area and run the recipe.sh + recompile. (Leave the old area so you can resubmit jobs from it) Otherwise, if you resubmit failed jobs from this updated area, the outputs for a single sample will have a mix of dataformats, which will cause weird crashes.

Evan

On Thu, Sep 27, 2012 at 2:45 PM, Tapas Sarangi notifications@github.comwrote:

do i need a recompilation after pulling the changes into local repo ?

— Reply to this email directly or view it on GitHubhttps://github.com/uwcms/FinalStateAnalysis/issues/11#issuecomment-8934806.

iross commented 11 years ago

Well I can't promise it shouldn't work. But I ran into a segfault when trying to run the "new" pattuple with old tags in my analysis code. I wouldn't try too hard to break things. I'll update the recipe_42x and push it in a few minutes..

tsarangi commented 11 years ago

Hi Evan,

Good to know.

On Sep 27, 2012, at 2:50 PM, Evan K. Friis wrote:

Hi Tapas,

I think anytime the PAT changes I would just create a new area and run the recipe.sh + recompile. (Leave the old area so you can resubmit jobs from it) Otherwise, if you resubmit failed jobs from this updated area, the outputs for a single sample will have a mix of dataformats, which will cause weird crashes.

But, doesn't this bring redundancies ? Do you need these new changes for the "VHlepdecay" samples. In any case, I haven't recompiled the new changes after taking them for the 53X samples. I am resubmitting some failed jobs for these, so it shouldn't be affected !

For 42x, I have already recompiled in the same repo, so oops, too late !

-Tapas

Evan

On Thu, Sep 27, 2012 at 2:45 PM, Tapas Sarangi notifications@github.comwrote:

do i need a recompilation after pulling the changes into local repo ?

— Reply to this email directly or view it on GitHubhttps://github.com/uwcms/FinalStateAnalysis/issues/11#issuecomment-8934806.

— Reply to this email directly or view it on GitHub.

ekfriis commented 11 years ago

@iross

But I ran into a segfault when trying to run the "new" pattuple with old tags

This is in general expected, but you should be able to read the new + old data with new tags.

@tsarangi

But, doesn't this bring redundancies ?

So I think the general rule is:

If there are changes to only data7TeV.py (or equivalent), ten you can pull into your current area (no recompiling necessary). If there are any changes to any thing in recipe_xx.sh, or anyting in PatTools/* then a new area should be created.

So for the 53X case there should be no problem to use the same area. For the 42X changes, since the recipe changes, a new area needs to be created.

Evan

tsarangi commented 11 years ago

If there are changes to only data7TeV.py (or equivalent), ten you can pull into your current area (no recompiling necessary). If there are any changes to any thing in recipe_xx.sh, or anyting in PatTools/* then a new area should be created.

I need to understand this a bit better. Creating a new area means doing "git clone ...." ? If yes, this becomes extremely difficult to keep track of ? I might be missing something trivial in here !

mcepeda commented 11 years ago

I believe by "new area" Evan means start from scratch a CMSSW release independent from the previous one? In a different location, and compile from scratch.

But that will simplify life, you will just end up with one CMSSW area per "big" request, that's all.

ekfriis commented 11 years ago

Hi Tapas,

Yes, exactly. I think for each "big" new production, you should give the CMSSW area the "name" of the PAT tuple.

scram pro -n 2012-09-27-PatTuple CMSSW CMSSW_5_2_5
cd 2012-09-27-PatTuple
git clone ....
recipe.sh #etc

And then never pull into this area if it would change the PAT tuple - this way you can always resubmit the jobs created in this area without things changing "midstream". It is a snapshot of the area when you created the PAT tuple.

Evan

tsarangi commented 11 years ago

8TeV VHlepdecay are done. Submitting 7 TeV sample with recent changes from Ian.

/hdfs/store/user/tapas/2012-09-18-8TeV-53X-PatTuple/VHtautaulepdecay*

tsarangi commented 11 years ago

Error (see below) while running ./install_python.sh for 4_2_8

I followed this page : http://final-state-analysis.readthedocs.org/en/latest/install.html

cloned this one : git clone git@github.com:uwcms/FinalStateAnalysis.git rather than this : https://ekfriis@github.com/ekfriis/FinalStateAnalysis.git

can this be the problem ? or there is anything I am missing.

Installing rootpy ./install_python.sh: line 17: cd: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/src/rootpy: No such file or directory python: can't open file 'setup.py': [Errno 2] No such file or directory ./install_python.sh: line 20: rehash: command not found Installing yolk ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory Installing PyYAML ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory ./install_python.sh: line 25: rehash: command not found Installing matplotlib ./install_python.sh: line 27: rehash: command not found ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory Installing ipython ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory Installing termcolor ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory Installing uncertainties <-- awesome error propagation ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory Install progressbar ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory Install cython ./install_python.sh: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTuple/src/FinalStateAnalysis/recipe/external/vpython/bin/pip: /afs/hep.wisc.edu/cms/tapas/FinalStateAnalysis_Package/2012-09-27-7TeV-PatTupl: bad interpreter: No such file or directory

ekfriis commented 11 years ago

Hi Tapas,

For PAT tuple production, you don't need to ./install_python.sh (only for analysis). The repo you cloned from is correct.

I will check why it is broken anyway....

E

tsarangi commented 11 years ago

7 TeV submitted.

login05:/scratch/tapas/2012-09-27-7TeV-PatTuple

ekfriis commented 11 years ago

Hi Tapas,

All the 7TeV jobs died (not sure about the 8TeV ones). I've found the problem - the code to extract the git commit ID crashed the job, since the "--no-shared-fs" is now the default for farmout, and it doesn't have access to the normal working area on AFS. This is fixed temporarily (at least it won't crash) in d9ee401, can you please try resubmitting the jobs?

As an aside, the --no-shared-fs feature is pretty cool for the PAT tuples. It tars up all the libs/python, so we can't have the problem any more of the working area changing out from under us. The problem is that then to fix a bug like this you need to create a new submit area (you can't rescue it).

Evan

ekfriis commented 11 years ago

FYI I'm stupid and had to add another commit to actually fix the problem... 75d37bc

tsarangi commented 11 years ago

7TeV samples submitted and running

ekfriis commented 11 years ago

Thanks, sorry for all the grief

On Mon, Oct 1, 2012 at 5:22 PM, Tapas Sarangi notifications@github.comwrote:

7TeV samples submitted and running

— Reply to this email directly or view it on GitHubhttps://github.com/uwcms/FinalStateAnalysis/issues/11#issuecomment-9035680.

tsarangi commented 11 years ago

Status : 7 TeV is also DONE

Output : /hdfs/store/user/tapas/2012-09-27-7TeV-PatTuple/

Closing this issue