search

GENIE-MC / Generator

The popular GENIE Generator product is used by nearly all accelerator neutrino experiments and it plays a key role in the exploitation of neutrino data. The Generator implements a modern software framework and it includes state-of-the-art physics modules. It captures the latest results of the GENIE global analysis of neutrino scattering data and includes several tunes that were produced using the proprietary Comparisons and Tuning products. The GENIE physics model is universal and comprehensive: It handles all neutrinos and targets, and all processes relevant from MeV to PeV energy scales. The Generator includes several tools (flux drivers, detector geometry navigators, specialized event generation apps, event reweighting engines) to simulate complex experimental setups in full detail and to support generator-related analysis tasks.
http://www.genie-mc.org
42 stars 92 forks source link

INCL++ ungracefully exits after a floating point error using gevgen #320

Open rdiurba opened 1 year ago

rdiurba commented 1 year ago

Whenever I run INCL++, the failure rate is high above a certain energy and the error message will not generate the event so I cannot use gevdump to investigate. I am using a slightly modified AR23_20i tune with INCL++, which I name AR23_20k. You can recreate this error with G18_11c.

Here are the setup commands I use:

source /grid/fermiapp/products/dune/setup_dune.sh
setup dk2nugenie v01_10_01k -q "e20:prof"
unsetup genie
setup genie v3_04_00d -q "e20:+inclxx:prof"

Here is the gevgen commands: gevgen -o testk1 -n 1000 -t 1000180400 --event-generator-list Default --tune AR23_20k_00_000 --cross-sections /pnfs/dune/persistent/users/rdiurba/gxspl-NUsmall_AR23_20k_00_000.xml -e 1 -p -14 --seed 1

And this is the error:

1692714574 NOTICE Nieves : [n] <NievesQELCCPXSec.cxx::LoadConfig (428)> : RPA=1, useCoulomb=1
1692714574 INFO TransverseEnhancementFFModel : [n] <ConfigIsotopeMapUtils.cxx::LoadAllNucARangesForKey (61)> : For 0 < A < 12 -> using MagFF_RT_A = 0
1692714574 INFO TransverseEnhancementFFModel : [n] <ConfigIsotopeMapUtils.cxx::LoadAllNucARangesForKey (61)> : For 11 < A < 420 -> using MagFF_RT_A = 5.19
1692714575 INFO TransverseEnhancementFFModel : [n] <ConfigIsotopeMapUtils.cxx::LoadAllNucARangesForKey (61)> : For 0 < A < 12 -> using MagFF_RT_B = 1
1692714575 INFO TransverseEnhancementFFModel : [n] <ConfigIsotopeMapUtils.cxx::LoadAllNucARangesForKey (61)> : For 11 < A < 420 -> using MagFF_RT_B = 0.376
1692714575 WARN IntLst : [n] <MECInteractionListGenerator.cxx::CreateInteractionList (55)> : fIsCC(val) = 1
1692714575 WARN IntLst : [n] <MECInteractionListGenerator.cxx::CreateInteractionList (99)> : fIsNC = 1
1692714575 WARN IntLst : [n] <MECInteractionListGenerator.cxx::CreateInteractionList (99)> : fIsNC = 1
1692714575 WARN IntLst : [n] <MECInteractionListGenerator.cxx::CreateInteractionList (99)> : fIsNC = 1
1692714575 NOTICE Ntp : [n] <NtpWriter.cxx::NtpWriter (45)> : Run number: 0
1692714575 NOTICE Ntp : [n] <NtpWriter.cxx::NtpWriter (46)> : Requested G/ROOT tree format: [NtpMCEventRecord]
1692714575 NOTICE Ntp : [n] <NtpMCJobConfig.cxx::Load (47)> : Converting configuration registries to TFolders
1692714575 NOTICE Ntp : [n] <NtpMCJobEnv.cxx::TakeSnapshot (46)> : Taking environment snapshot and saving it in a TFolder
1692714575 NOTICE HINCLCascadeIntranuke : [n] <HINCLCascadeIntranuke.cxx::ProcessEventRecord (459)> : ************ Running HINCLCascadeIntranuke MODE INTRANUKE ************
1692714575 NOTICE HINCLCascadeIntranuke : [n] <HINCLCascadeIntranuke.cxx::TransportHadrons (500)> : Propagating hadrons within nucleus found in position = 3
1692714575 NOTICE HINCLCascadeIntranuke : [n] <HINCLCascadeIntranuke.cxx::TransportHadrons (516)> : Nucleus (A,Z) = (39, 18)
1692714575 NOTICE HINCLCascadeIntranuke : [n] <HINCLCascadeIntranuke.cxx::ProcessEventRecord (459)> : ************ Running HINCLCascadeIntranuke MODE INTRANUKE ************
1692714575 NOTICE HINCLCascadeIntranuke : [n] <HINCLCascadeIntranuke.cxx::TransportHadrons (500)> : Propagating hadrons within nucleus found in position = 3
1692714575 NOTICE HINCLCascadeIntranuke : [n] <HINCLCascadeIntranuke.cxx::TransportHadrons (516)> : Nucleus (A,Z) = (39, 18)
Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |     nu_mu_bar |            -14 |      0 |  -1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  1.000 |  1.000 |  0.000 |  
ERROR - Floating point invalid operation.
Floating point exception
dytman commented 12 months ago

thanks for posting this. I think I see similar problem, was able to narrow it down to QE and MEC. It's not a problem with RES or DIS. Can you narrow problem down to a specific process? Error messaging isn't helpful here.

rdiurba commented 12 months ago

I used the same command as above but changed --event-generator-list to CCMEC. I was able to recreate the error with the same seed. Here is the printout on Messenger_rambling.xml

1693929040 WARN IntLst : [n] <MECInteractionListGenerator.cxx::CreateInteractionList (55)> : fIsCC(val) = 1
1693929040 NOTICE Ntp : [n] <NtpWriter.cxx::NtpWriter (45)> : Run number: 0
1693929040 NOTICE Ntp : [n] <NtpWriter.cxx::NtpWriter (46)> : Requested G/ROOT tree format: [NtpMCEventRecord]
1693929040 NOTICE Ntp : [n] <NtpMCJobConfig.cxx::Load (47)> : Converting configuration registries to TFolders
1693929040 NOTICE Ntp : [n] <NtpMCJobEnv.cxx::TakeSnapshot (46)> : Taking environment snapshot and saving it in a TFolder
1693929040 NOTICE gevgen : [n] <gEvGen.cxx::GenerateEventsAtFixedInitState (312)> : 
 ** Will generate 1000 events for 
[-] [Init-State] 
 |--> probe        : PDG-code = -14 (nu_mu_bar)
 |--> nucl. target : Z = 18, A = 40, PDG-Code = 1000180400 (Ar40)
 |--> hit nucleon  : no set
 |--> hit quark    : no set
 |--> probe 4P     : (E =            0, Px =            0, Py =            0, Pz =            0)
 |--> target 4P    : (E =      37.2155, Px =            0, Py =            0, Pz =            0)
 at Ev = 1 GeV
1693929040 NOTICE gevgen : [n] <gEvGen.cxx::GenerateEventsAtFixedInitState (319)> :  *** Generating event............ 0
1693929040 NOTICE IntSel : [n] <PhysInteractionSelector.cxx::SelectInteraction (86)> : 
======================================================================================================
  Selecting an interaction for the given initial state = nu-pdg:-14;tgt-pdg:1000180400; at E = 1 GeV  
======================================================================================================
1693929040 NOTICE IntSel : [n] <PhysInteractionSelector.cxx::SelectInteraction (88)> : Computing xsecs for all relevant modeled interactions:
1693929040 NOTICE IntSel : [n] <PhysInteractionSelector.cxx::SelectInteraction (154)> : 
 |---------------------------------------------------------------------------------------------------------------|
 |                                                                      interaction | cross-section (1E-38*cm^2) |
 |---------------------------------------------------------------------------------------------------------------|
 |                                         nu:-14;tgt:1000180400;proc:Weak[CC],MEC; |                    1.66648 | 
 |---------------------------------------------------------------------------------------------------------------|

1693929040 NOTICE IntSel : [n] <PhysInteractionSelector.cxx::SelectInteraction (190)> : Selected interaction: nu:-14;tgt:1000180400;proc:Weak[CC],MEC;
1693929040 NOTICE GEVGDriver : [n] <GEVGDriver.cxx::GenerateEvent (282)> : 
==============================================================================================================
  Requesting from event generation thread: genie::EventGenerator/MEC-CC to generate the selected interaction  
==============================================================================================================

1693929040 NOTICE EventGenerator : [n] <EventGenerator.cxx::ProcessEventRecord (66)> : Generating Event...
1693929040 NOTICE EventGenerator : [n] <EventGenerator.cxx::ProcessEventRecord (69)> : Clearing the GHepVirtualListFolder
1693929040 NOTICE EventGenerator : [n] <EventGenerator.cxx::ProcessEventRecord (97)> : 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Event generation thread: genie::EventGenerator/MEC-CC -> Running module: genie::InitialStateAppender/Default  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1693929040 INFO ISApp : [n] <InitialStateAppender.cxx::ProcessEventRecord (48)> : Adding the initial state to the event record
1693929040 INFO ISApp : [n] <InitialStateAppender.cxx::AddNeutrino (76)> : Adding neutrino [pdgc = -14]
1693929040 INFO ISApp : [n] <InitialStateAppender.cxx::AddNucleus (100)> : Adding nucleus [A = 40, Z = 18, pdg = 1000180400]
1693929040 INFO ISApp : [n] <InitialStateAppender.cxx::ProcessEventRecord (63)> : 

|------------------------------------------------------------------------------------------------------------------|
|GENIE GHEP Event Record [print level:   3]                                                                        |
|------------------------------------------------------------------------------------------------------------------|
| Idx |          Name | Ist |        PDG |   Mother  | Daughter  |      Px |      Py |      Pz |       E |      m  | 
|------------------------------------------------------------------------------------------------------------------|
|   0 |     nu_mu_bar |   0 |        -14 |  -1 |  -1 |  -1 |  -1 |   0.000 |   0.000 |   1.000 |   1.000 |   0.000 | 
|   1 |          Ar40 |   0 | 1000180400 |  -1 |  -1 |  -1 |  -1 |   0.000 |   0.000 |   0.000 |  37.216 |  37.216 | 
|------------------------------------------------------------------------------------------------------------------|
|       Fin-Init:                                                |   0.000 |   0.000 |  -1.000 | -38.216 |         | 
|------------------------------------------------------------------------------------------------------------------|
|       Vertex:      nu_mu_bar @ (x =     0.00000 m, y =     0.00000 m, z =     0.00000 m, t =    0.000000e+00 s)  |
|------------------------------------------------------------------------------------------------------------------|
| Err flag [bits:15->0] : 0000000000000000    |  1st set:                                                     none | 
| Err mask [bits:15->0] : 1111111111111111    |  Is unphysical:    NO |   Accepted:   YES                          |
|------------------------------------------------------------------------------------------------------------------|
| sig(Ev) =       1.66648e-38 cm^2  | dsig(Ev;{K_s})/dK   =     0.00000e+00 cm^2/{K}   | Weight =          1.00000 |
|------------------------------------------------------------------------------------------------------------------|

--------------------------------------------------------------------------------------------------------------
GENIE Interaction Summary
--------------------------------------------------------------------------------------------------------------
[-] [Init-State] 
 |--> probe        : PDG-code = -14 (nu_mu_bar)
 |--> nucl. target : Z = 18, A = 40, PDG-Code = 1000180400 (Ar40)
 |--> hit nucleon  : no set
 |--> hit quark    : no set
 |--> probe 4P     : (E =     1.000000, Px =     0.000000, Py =     0.000000, Pz =     1.000000)
 |--> target 4P    : (E =    37.215526, Px =     0.000000, Py =     0.000000, Pz =     0.000000)

[-] [Process-Info]  
 |--> Interaction : Weak[CC]
 |--> Scattering  : MEC
[-] [Kinematics]
[-] [Exclusive Process Info] 
 |--> charm prod.  : false |--> strange prod.  : false
 |--> f/s nucleons : N(p) = 0 N(n) = 0
 |--> f/s pions    : N(pi^0) = 0 N(pi^+) = 0 N(pi^-) = 0
 |--> f/s Other    : N(gamma) = 0 N(Rho^0) = 0 N(Rho^+) = 0 N(Rho^-) = 0
 |--> resonance    : [not set]
 |--> final quark prod.  : false
 |--> final lepton prod.  : false
--------------------------------------------------------------------------------------------------------------

1693929040 NOTICE EventGenerator : [n] <EventGenerator.cxx::ProcessEventRecord (97)> : 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Event generation thread: genie::EventGenerator/MEC-CC -> Running module: genie::VertexGenerator/Default  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1693929040 INFO Vtx : [n] <VertexGenerator.cxx::GenerateVertex (146)> : Generating vertex according to a realistic nuclear density profile
1693929040 INFO Vtx : [n] <VertexGenerator.cxx::GenerateVertex (211)> : Generated vtx @ r = 1.84035 fm / (x = 1.30244, y = -0.00785201, z = -1.30019)
1693929040 INFO Vtx : [n] <VertexGenerator.cxx::ProcessEventRecord (73)> : Setting vertex position for: nu_mu_bar
1693929040 NOTICE EventGenerator : [n] <EventGenerator.cxx::ProcessEventRecord (97)> : 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Event generation thread: genie::EventGenerator/MEC-CC -> Running module: genie::MECGenerator/Default  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Collection name='genie::GHepParticles', class='genie::EventRecord', size=1000

 |     nu_mu_bar |            -14 |      0 |  -1 |  -1 |  -1 |  -1 |  0.000 |  0.000 |  1.000 |  1.000 |  0.000 |  
ERROR - Floating point invalid operation.
Floating point exception
dytman commented 12 months ago

yes, that is the idea. Use CCQE, CCMEC, CCRES, and CCDIS separately to show which simulations are affected. By the way, I use G18_10c events on my laptop so it's likely more general. To be specific, problem seems to be with 1st event of a specific process.

rdiurba commented 12 months ago

I am sorry but I do not know if I have access to a G18_10a xml spline, hence I am using AR23_20i splines. I have tried all CC topologies and they all work except for CCMEC.

mroda88 commented 9 months ago

The fix should be in #360

rdiurba commented 9 months ago

Hey Marco,

Thank you for the update. I have a few questions and then I will try it out in a bit.

Are there any additional fixes planned or does this extend from the MEC issue to the O11 issue Julia and I saw? Is a tagged release planned in the near future?

Thank you to the whole GENIE team for all the hard work!

Thanks,

Richie

From: Marco Roda @.> Sent: Thursday, November 9, 2023 2:54 PM To: GENIE-MC/Generator @.> Cc: rdiurba @.>; Author @.> Subject: Re: [GENIE-MC/Generator] INCL++ ungracefully exits after a floating point error using gevgen (Issue #320)

The fix should be in #360https://github.com/GENIE-MC/Generator/pull/360

— Reply to this email directly, view it on GitHubhttps://github.com/GENIE-MC/Generator/issues/320#issuecomment-1803876512, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOZJTLFBY3TSOGYAPGUEX5LYDTODBAVCNFSM6AAAAAA32BAJHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBTHA3TMNJRGI. You are receiving this because you authored the thread.Message ID: @.***>

mroda88 commented 9 months ago

This should fix everything. Although we have yet to test it.

We think we will release a new tag before the Christmas holidays. We know for sure that the next release will require this fix anyway.

rdiurba commented 9 months ago

Hey Marco and GENIE,

I ran some CCMEC events at 5 GeV for AR23 with the three different FSIs and I was able to run CCMEC on INCL++, which is a good sign. Furthermore, the double-banding for Geant4 that Callum saw seems to be fixed in the images attached.

I was able to run gevgen_had on carbon, iron, and lead on INCL++ and Geant4. I did not see any bugs, whereas before carbon and lead would fail due to various isotopes being generated.

I will keep investigating but so far progress has been made.

Thanks,

Richie

From: Marco Roda @.> Sent: Thursday, November 9, 2023 3:11 PM To: GENIE-MC/Generator @.> Cc: rdiurba @.>; Author @.> Subject: Re: [GENIE-MC/Generator] INCL++ ungracefully exits after a floating point error using gevgen (Issue #320)

This should fix everything. Although we have yet to test it.

We think we will release a new tag before the Christmas holidays. We know for sure that the next release will require this fix anyway.

— Reply to this email directly, view it on GitHubhttps://github.com/GENIE-MC/Generator/issues/320#issuecomment-1803907413, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOZJTLHTVWOBQ64KLZTTE3TYDTQB3AVCNFSM6AAAAAA32BAJHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBTHEYDONBRGM. You are receiving this because you authored the thread.