search

cms-sw / genproductions

Generator fragments for MC production
https://twiki.cern.ch/twiki/bin/view/CMS/GitRepositoryForGenProduction
79 stars 786 forks source link

cleangridmore.sh breaks MadSpin #384

Closed emanueleusai closed 9 years ago

emanueleusai commented 9 years ago

Hello,

I'm producing gridpacks using MadGraph+MadSpin. After I produce a gridpack, if I try to generate events with it, MadSpin complains about missing fortran source files in mgbasedir/models/template_files/fortran/, mgbasedir/aloha/template_files/, and possibly other folders. If I remove the line "find ./ -name "*.f" | xargs -r rm" in cleangridmore.sh and re-produce the gridpack, then the generation works correctly. However this might impact negatively on the size of the gridpack in case a large number of diagrams needs to be computed.

Cheers, Emanuele

bendavid commented 9 years ago

This likely was just due to a failure in the madspin generation in the first attempt. In this case the missing files will trigger a failure later because the gridpack run will try to recompile madspin.

In case madspin succeeds on the first try, these files should really not be needed. Can you try one more time to make the gridpack with the cleangridmore.sh included? The fact that it worked the second time and not the first may just be a coincidence.

emanueleusai commented 9 years ago

Hi Josh,

Thanks for addressing my issue. I've tried a second time using the untouched cleangridmore.sh. Here is the generated gridpack: https://cernbox.cern.ch/public.php?service=files&t=56b2f95361c7aeb4281831c81a43c2b0 During gridpack production MadSpin seems to be working fine:

************************************************************
*                                                          *
*           W E L C O M E  to  M A D S P I N               *
*                                                          *
************************************************************
MadSpin>INFO: process: p p > rho0 t23 > t~ t h 
INFO: options:  
Set group_subprocesses to Auto
Note that you need to regenerate all processes
set loop optimized output to True
INFO: detected model: onerho. Loading... 
INFO: load particles 
INFO: load vertices 
INFO: Change particles name to pass to MG5 convention 
Defined multiparticle p = g u c d s u~ c~ d~ s~
Defined multiparticle j = g u c d s u~ c~ d~ s~
Defined multiparticle l+ = e+ mu+
Defined multiparticle l- = e- mu-
Defined multiparticle vl = ve vm vt
Defined multiparticle vl~ = ve~ vm~ vt~
Defined multiparticle all = g u c d s u~ c~ d~ s~ a ve vm vt e- mu- ve~ vm~ vt~ e+ mu+ t b t23 b13 x53 x23 t~ b~ t23~ b13~ x53~ x23~ z w+ h rho+ rho0 w- rho- ta- ta+
Set group_subprocesses to Auto
Note that you need to regenerate all processes
set loop optimized output to True
Complex mass already desactivated.
Defined multiparticle p = g u c d s u~ c~ d~ s~
Defined multiparticle j = g u c d s u~ c~ d~ s~
Defined multiparticle l+ = e+ mu+
Defined multiparticle l- = e- mu-
Defined multiparticle vl = ve vm vt
Defined multiparticle vl~ = ve~ vm~ vt~
Defined multiparticle bb = b b~
Defined multiparticle ww = w+ w-
Defined multiparticle tt = t t~
Defined multiparticle ferm = u c d s u~ c~ d~ s~ ve vm vt e- mu- ve~ vm~ vt~ e+ mu+ b b~ ta- ta+
Defined multiparticle topo = t23 t23~
MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>INFO: Will use seed 851040630 
INFO: We need to recalculate the branching fractions for t~,w-,w+,t 
INFO: using the FeynRules formula present in the model (arXiv:1402.1178) 
INFO:    
INFO: decay channels for t~ : ( width = 1.466877 GeV ) 
INFO:        BR                 d1  d2 
INFO:    1.000000e+00            b~  w-  
INFO:    
INFO:    
INFO: decay channels for w- : ( width = 2.00278 GeV ) 
INFO:        BR                 d1  d2 
INFO:    3.333609e-01            s  c~  
INFO:    3.333609e-01            d  u~  
INFO:    1.111203e-01            mu-  vm~  
INFO:    1.111203e-01            e-  ve~  
INFO:    1.110377e-01            ta-  vt~  
INFO:    
INFO:    
INFO: decay channels for w+ : ( width = 2.00278 GeV ) 
INFO:        BR                 d1  d2 
INFO:    3.333609e-01            s~  c  
INFO:    3.333609e-01            d~  u  
INFO:    1.111203e-01            mu+  vm  
INFO:    1.111203e-01            e+  ve  
INFO:    1.110377e-01            ta+  vt  
INFO:    
INFO:    
INFO: decay channels for t : ( width = 1.466877 GeV ) 
INFO:        BR                 d1  d2 
INFO:    1.000000e+00            b  w+  
INFO:    
INFO: generating the production square matrix element 
INFO: generate p p > rho0 t23 > t~ t h  --no_warning=duplicate;add process p p > rho0 t23~ > t~ t h  --no_warning=duplicate; 
INFO: Done 5.699 
INFO: generating the full square matrix element (with decay) 
INFO: generate p p > rho0 t23 > t~ t h, (t~ > b~ w- , w- > all all  QCD=99), (t > b w+ , w+ > all all  QCD=99)   --no_warning=duplicate;add process p p > rho0 t23~ > t~ t h, (t~ > b~ w- , w- > all all  QCD=99), (t > b w+ , w+ > all all  QCD=99)   --no_warning=duplicate; 
INFO: Done 19.16 
INFO: generate matrix element for decay only (1 - > N). 
INFO: output standalone_msF /afs/cern.ch/work/e/eusai/public/gen/genproductions/bin/MadGraph5_aMCatNLO/Ht/Ht_gridpack/work/decay_me 
INFO: Done 7.42 
INFO: Compiling code 
INFO: detect independant decays 
INFO: Done in 0.274948835373s 
INFO:    
INFO:    Estimating the maximum weight     
INFO:    *****************************     
INFO:      Probing the first 75 events 
INFO:      with 400 phase space points 
INFO:    
INFO: Event 1/75 :  0.47s   
INFO: Event 6/75 :  2.1s   
INFO: Event 11/75 :  3.5s   
INFO: Event 16/75 :  4.9s   
INFO: Event 21/75 :  6.3s   
INFO: Event 26/75 :  7.7s   
INFO: Event 31/75 :  9.2s   
INFO: Event 36/75 :  10.6s   
INFO: Event 41/75 :  12.2s   
INFO: Event 46/75 :  13.7s   
INFO: Event 51/75 :  15.2s   
INFO: Event 56/75 :  16.7s   
INFO: Event 61/75 :  18.2s   
INFO: Event 66/75 :  19.8s   
INFO: Event 71/75 :  21.1s   
INFO:   
INFO: Decaying the events...  
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Got more than 2 (3) daughters for one particles 
INFO: in one production event (before decay) 
INFO: Total number of events written: 776/776  
INFO: Average number of trial points per production event: 9.77963917526 
INFO: Branching ratio to allowed decays: 1 
INFO: Number of events with weights larger than max_weight: 0 
INFO: Number of subprocesses 8 
INFO: Number of failures when restoring the Monte Carlo masses: 0  
INFO: Decayed events have been written in /afs/cern.ch/work/e/eusai/public/gen/genproductions/bin/MadGraph5_aMCatNLO/Ht/Ht_gridpack/work/unweighted_events_decayed.lhe.gz 
MadSpin> 

preparing final gridpack
cleaning unneeded files from gridpack
Saving log file
Creating tarball
Gridpack created successfully at /afs/cern.ch/user/e/eusai/work/public/gen/genproductions/bin/MadGraph5_aMCatNLO/Ht_tarball.tar.xz
End of job

However when I try to generate events I get again:

bash-4.1$ ./runcmsgrid.sh 10000 1 1
%MSG-MG5 number of events requested = 10000
%MSG-MG5 random seed used for the run = 1
%MSG-MG5 number of cpus = 1
Updating grid_card.dat...
Now generating 10000 events with random seed 1 and granularity -1
P0_qq_txth
DONE
Moving events from  events.lhe
************************************************************
*                                                          *
*           W E L C O M E  to  M A D S P I N               *
*                                                          *
************************************************************
MadSpin>INFO: process: p p > rho0 t23 > t~ t h 
INFO: options:  
Set group_subprocesses to Auto
Note that you need to regenerate all processes
set loop optimized output to True
INFO: detected model: onerho. Loading... 
INFO: reload from .py file 
INFO: load particles 
INFO: load vertices 
INFO: Change particles name to pass to MG5 convention 
Defined multiparticle p = g u c d s u~ c~ d~ s~
Defined multiparticle j = g u c d s u~ c~ d~ s~
Defined multiparticle l+ = e+ mu+
Defined multiparticle l- = e- mu-
Defined multiparticle vl = ve vm vt
Defined multiparticle vl~ = ve~ vm~ vt~
Defined multiparticle all = g u c d s u~ c~ d~ s~ a ve vm vt e- mu- ve~ vm~ vt~ e+ mu+ t b t23 b13 x53 x23 t~ b~ t23~ b13~ x53~ x23~ z w+ h rho+ rho0 w- rho- ta- ta+
Set group_subprocesses to Auto
Note that you need to regenerate all processes
set loop optimized output to True
Complex mass already desactivated.
Defined multiparticle p = g u c d s u~ c~ d~ s~
Defined multiparticle j = g u c d s u~ c~ d~ s~
Defined multiparticle l+ = e+ mu+
Defined multiparticle l- = e- mu-
Defined multiparticle vl = ve vm vt
Defined multiparticle vl~ = ve~ vm~ vt~
Defined multiparticle bb = b b~
Defined multiparticle ww = w+ w-
Defined multiparticle tt = t t~
Defined multiparticle ferm = u c d s u~ c~ d~ s~ ve vm vt e- mu- ve~ vm~ vt~ e+ mu+ b b~ ta- ta+
Defined multiparticle topo = t23 t23~
MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>MadSpin>INFO: We need to recalculate the branching fractions for t~,w-,w+,t 
INFO: using the FeynRules formula present in the model (arXiv:1402.1178) 
INFO:    
INFO: decay channels for t~ : ( width = 1.466877 GeV ) 
INFO:        BR                 d1  d2 
INFO:    1.000000e+00            b~  w-  
INFO:    
INFO:    
INFO: decay channels for w- : ( width = 2.00278 GeV ) 
INFO:        BR                 d1  d2 
INFO:    3.333609e-01            s  c~  
INFO:    3.333609e-01            d  u~  
INFO:    1.111203e-01            mu-  vm~  
INFO:    1.111203e-01            e-  ve~  
INFO:    1.110377e-01            ta-  vt~  
INFO:    
INFO:    
INFO: decay channels for w+ : ( width = 2.00278 GeV ) 
INFO:        BR                 d1  d2 
INFO:    3.333609e-01            s~  c  
INFO:    3.333609e-01            d~  u  
INFO:    1.111203e-01            mu+  vm  
INFO:    1.111203e-01            e+  ve  
INFO:    1.110377e-01            ta+  vt  
INFO:    
INFO:    
INFO: decay channels for t : ( width = 1.466877 GeV ) 
INFO:        BR                 d1  d2 
INFO:    1.000000e+00            b  w+  
INFO:    
INFO: generating the production square matrix element 
INFO: generate p p > rho0 t23 > t~ t h  --no_warning=duplicate;add process p p > rho0 t23~ > t~ t h  --no_warning=duplicate; 
Command "launch" interrupted with error:
OSError : [Errno 2] No such file or directory: '/afs/cern.ch/work/e/eusai/public/gen/genproductions/bin/MadGraph5_aMCatNLO/testproduction/CMSSW_7_1_14/src/mgbasedir/models/template_files/fortran/printout.f'
Please report this bug to developers

           More information is found in 'MS_debug'.

           Please attach this file to your report.
MadSpin> 

mv: cannot stat `process/events_decayed.lhe.gz': No such file or directory
gzip: events_presys.lhe.gz: No such file or directory

Any idea on what might be wrong in my setup?

bendavid commented 9 years ago

It looks like your gridpack is not reusing the madspin directory which should be generated at gridpack creation time.

Do you have the "set ms_dir" in your madspin cards as in the example?

https://github.com/cms-sw/genproductions/blob/master/bin/MadGraph5_aMCatNLO/cards/examples/wplustest_5f_NLO/wplustest_5f_NLO_madspin_card.dat

emanueleusai commented 9 years ago

Problem solved. That was the line missing. Sorry, I blindly copied the madspin card from my private tests in gridpack-less mode

Thank you very much for the help!

bendavid commented 9 years ago

ok great.

Make sure you also have the max_running_processes line https://github.com/cms-sw/genproductions/blob/master/bin/MadGraph5_aMCatNLO/cards/examples/wplustest_5f_NLO/wplustest_5f_NLO_madspin_card.dat#L22

Since otherwise the grid jobs can run into batch worker limits and crash.