search

GENIE-MC / Generator

The popular GENIE Generator product is used by nearly all accelerator neutrino experiments and it plays a key role in the exploitation of neutrino data. The Generator implements a modern software framework and it includes state-of-the-art physics modules. It captures the latest results of the GENIE global analysis of neutrino scattering data and includes several tunes that were produced using the proprietary Comparisons and Tuning products. The GENIE physics model is universal and comprehensive: It handles all neutrinos and targets, and all processes relevant from MeV to PeV energy scales. The Generator includes several tools (flux drivers, detector geometry navigators, specialized event generation apps, event reweighting engines) to simulate complex experimental setups in full detail and to support generator-related analysis tasks.
http://www.genie-mc.org
42 stars 91 forks source link

Broken GEM21_11a configuration #395

Closed jtenavidal closed 3 months ago

jtenavidal commented 3 months ago

It seems the latest EM merge broke the GEM21_11a_00_000 tune. The code I run is the following: 

gevgen -p 11 -n 300000 -e 4.325 -t 1000060120 -r 1110000100100 --seed 1210931039 --cross-sections [GEM21_11a_Dipole_master_eNuclei.txt](https://github.com/GENIE-MC/Generator/files/14746548/GEM21_11a_Dipole_master_eNuclei.txt) --event-generator-list EMQE --tune GEM21_11a_00_000

(Spline attached in txt format)

I get many errors like the following: 

1711382916 INFO TabulatedHadronTensorModelI : [n] <TabulatedHadronTensorModelI.cxx::BuildTensor (312)> : Loading the hadron tensor data file /genie/app/users/jtenavid/Software/genie-v3/Generator/src/scripts/production/python/Generator/data/evgen/hadron_tensors/crpa_susav2/SuSAv2_1000060120_QE_EM.dat
TabulatedLabFrameHadronTensor: reading hadron tensor table
table_file_name:  /genie/app/users/jtenavid/Software/genie-v3/Generator/src/scripts/production/python/Generator/data/evgen/hadron_tensors/crpa_susav2/SuSAv2_1000060120_QE_EM.dat
Z:                6
A:                12
num_q0:           401
num_q_mag:        401
1711382919 ERROR PDG : [n] <PDGLibrary.cxx::Find (93)> : Requested missing particle with PDG: 1000040040
1711382925 ERROR PDG : [n] <PDGLibrary.cxx::Find (93)> : Requested missing particle with PDG: 1000000030
1711382981 ERROR PDG : [n] <PDGLibrary.cxx::Find (93)> : Requested missing particle with PDG: 1000000030

Now my job always gets on hold at the grid due to exceeding the limit time.

The generation seems very slow. For MEC It prints out a lot of information:

Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |   3 |   3 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |   2 |   2 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 |  
 |           p+p |     2000000202 |     11 |   1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 |  0.000 |  1.877 |  
 |            e- |             11 |      1 |   0 |  -1 |  -1 |  -1 | -0.205 |  0.155 |  4.030 |  4.038 |  0.001 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |   4 |   4 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |   2 |   3 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 |  
 |           p+p |     2000000202 |     11 |   1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 |  0.000 |  1.877 |  
 |          Be10 |     1000040100 |      1 |   1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 | 11.175 |  9.326 |  
 |            e- |             11 |      1 |   0 |  -1 |  -1 |  -1 | -0.205 |  0.155 |  4.030 |  4.038 |  0.001 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |   4 |   4 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |   2 |   3 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 |  
 |           p+p |     2000000202 |     11 |   1 |  -1 |   5 |   5 | -0.087 |  0.214 | -0.148 |  1.897 |  1.877 |  
 |          Be10 |     1000040100 |      1 |   1 |  -1 |  -1 |  -1 |  0.087 | -0.214 |  0.148 |  9.318 |  9.326 |  
 |            e- |             11 |      1 |   0 |  -1 |  -1 |  -1 | -0.205 |  0.155 |  4.030 |  4.038 |  0.001 |  
 |           p+p |     2000000202 |      3 |   2 |  -1 |  -1 |  -1 |  0.118 |  0.059 |  0.147 |  2.144 |  1.877 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |   3 |   3 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |   2 |   2 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 |  
 |           n+p |     2000000201 |     11 |   1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 |  0.000 |  1.878 |  
 |            e- |             11 |      1 |   0 |  -1 |  -1 |  -1 | -0.293 |  0.119 |  4.283 |  4.295 |  0.001 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |   4 |   4 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |   2 |   3 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 |  
 |           n+p |     2000000201 |     11 |   1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 |  0.000 |  1.878 |  
 |           B10 |     1000050100 |      1 |   1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 | 11.175 |  9.324 |  
 |            e- |             11 |      1 |   0 |  -1 |  -1 |  -1 | -0.293 |  0.119 |  4.283 |  4.295 |  0.001 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |   4 |   4 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |   2 |   3 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 |  
 |           n+p |     2000000201 |     11 |   1 |  -1 |   5 |   5 | -0.254 |  0.043 | -0.056 |  1.896 |  1.878 |  
 |           B10 |     1000050100 |      1 |   1 |  -1 |  -1 |  -1 |  0.254 | -0.043 |  0.056 |  9.319 |  9.324 |  
 |            e- |             11 |      1 |   0 |  -1 |  -1 |  -1 | -0.293 |  0.119 |  4.283 |  4.295 |  0.001 |  
 |           n+p |     2000000201 |      3 |   2 |  -1 |  -1 |  -1 |  0.039 | -0.076 | -0.014 |  1.886 |  1.878 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |            e- |             11 |      0 |  -1 |  -1 |   3 |   3 |  0.000 |  0.000 |  4.325 |  4.325 |  0.001 |  
 |           C12 |     1000060120 |      0 |  -1 |  -1 |   2 |   2 |  0.000 |  0.000 |  0.000 | 11.175 | 11.175 |  
 |           n+n |     2000000200 |     11 |   1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  0.000 |  0.000 |  1.879 |  
 |            e- |             11 |      1 |   0 |  -1 |  -1 |  -1 |  0.142 | -0.129 |  4.236 |  4.240 |  0.001 | Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

Can we turn this off? Sometimes the output gives me errors

EMQE: it is quite fast. I get no odd messages

The rest is as fast as for G18

idkakorin commented 3 months ago

Julia,

I get many errors like the following: 1711382919 ERROR PDG : [n] <PDGLibrary.cxx::Find (93)> : Requested missing particle with PDG: 1000040040 1711382925 ERROR PDG : [n] <PDGLibrary.cxx::Find (93)> : Requested missing particle with PDG: 1000000030 1711382981 ERROR PDG : [n] <PDGLibrary.cxx::Find (93)> : Requested missing particle with PDG: 1000000030

These error messages are already printed in version 3.4.0. They are generated by PDGLibrary.cxx, which are called by HAIntranuke2018.cxx. Both files weren't essentially changed for a long time and so it is not quite clear, what do you mean by "the latest EM merge", could you clarify it, please?

idkakorin commented 3 months ago

What concerns the second issue:

Now my job always gets on hold at the grid due to exceeding the limit time. The generation seems very slow. For MEC It prints out a lot of information:

Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

| e- | 11 | 0 | -1 | -1 | 3 | 3 | 0.000 | 0.000 | 4.325 | 4.325 | 0.001 |
| C12 | 1000060120 | 0 | -1 | -1 | 2 | 2 | 0.000 | 0.000 | 0.000 | 11.175 | 11.175 |
| p+p | 2000000202 | 11 | 1 | -1 | -1 | -1 | 0.000 | 0.000 | 0.000 | 0.000 | 1.877 |
| e- | 11 | 1 | 0 | -1 | -1 | -1 | -0.205 | 0.155 | 4.030 | 4.038 | 0.00

The outputs are controlled by configuration in Messenger.xml files. You can use settings from v.3.4.0 for main branch or just set another message threshold.

Approximate processing time/event on my laptop (for 50000 events) are 0.0100529 s for master branch and 0.00974255 s for v.3.4.0. They differ less than 3.5% due to different priority for gevgen message stream. This difference is not critical. Therefore, it is strange why now your jobs always get on hold at the grid due to exceeding the limit time, but you didn't observe the analogous problem with GENIE v.3.4.0. And again, it is not clear why "the latest EM merge" is in blame?

jtenavidal commented 3 months ago

@idkakorin I am still investigating it but yes, before I could run jobs without issue but since few weeks I seem to get them all on hold. I will have a look at the Messenger.xml, I turned off most of the flags but it might be there is a new one. Let's close this issue for now if those errors are expected

idkakorin commented 3 months ago

@jtenavidal

Let's close this issue for now if those errors are expected

Ok, but just one comment. The errors like "Requested missing particle with PDG: 1000040040" are due to HAIntranuke2018.cxx. As I understand, they arise if the remnant nucleus can't be found in PDG table and HardBlob is formed at the end. I am not pretty sure, that this behavior is normal, but it is decently is not related to the "the latest EM merge". Maybe it is worth to consult with Steve Dytman?

I will have a look at the Messenger.xml

I would suggest using Messenger_whisper.xml which suppress almost all messages.