Segmentation Fault when running ShipReco.py

[mferril]

Hi @olantwin @anupama-reghunath

FYI: I have also seen some unexpected behaviour when launching ShipReco.py on just 2 events, as a test. The output file has been successfully created, it's uncorrupted. However the script ends in Segmentation violation:

[FairShip/latest] FairShip/macro $> python ShipReco.py -f test/ship.conical.EvtCalc-TGeant4.root -g test/geofile_full.conical.EvtCalc-TGeant4.root -n 2
configured to process  2  events from  test/ship.conical.EvtCalc-TGeant4.root  starting with event  0  with option Yheight =  None  with vertexing True  and real pattern reco  
[INFO] Media file used: sw/slc9_x86-64/FairShip/master-local1/geometry/media.geo
Info in <ecal::ecal>: Geometry is read from file ecal_rect5x10m2.geoz3613.6000000000004.geo.
Info in <ecal::ecal>: Size of cell of type 2 is 5.970000 cm.
Info in <hcal::hcal>: Geometry is read from file hcal_rect.geoz3713.6000000000004.geo.
Info in <hcal::hcal>: Number of modules is 924, lightmap none
Info in <hcal::hcal>: Size of cell of type 22 is 23.959999 cm.
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CountLevels>: max level = 6, max placements = 897
Info in <TGeoManager::CloseGeometry>: 5342106 nodes/ 728 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Calling addVMCFields
Creating field map called MainSpecMap using sw/slc9_x86-64/FairShip/master-local1/files/MainSpectrometerField.root
ShipBFieldMap::readMapFile() creating field MainSpecMap using file sw/slc9_x86-64/FairShip/master-local1/files/MainSpectrometerField.root
x limits: -470, 470, dx = 10
y limits: -405, 405, dy = 10
z limits: -990, 990, dz = 10
Offsets: x = 0, y = 0, z = 3068
Angles : phi = 0, theta = 0, psi = 0
Total number of bins = 1550210; Nx = 95, Ny = 82, Nz = 199
Setting the Global field
Adding field MainSpecMap to Global
The virtual MC pointer gMC is null! The global field can't be used by Geant4 but will work for track fitting and track extrapolation
****************************************
*** You are using PID version 18.1.0 ***
****************************************
** initialize Calo reconstruction **
-I- ecalStructure::Construct(): calorimeter matrix created.
event  0
TStreamerInfo::Build:0: RuntimeWarning: genfit::TrackPoint: boost::scoped_ptr<genfit::ThinScatterer> has no streamer or dictionary, data member "thinScatterer_" will not be saved
finished writing tree
Exit normally
 *** Break *** segmentation violation

Running in --Debug mode does not add any useful piece of info, unfortunately.

[olantwin]

Debugging this seems quite non-trivial:

• We know that it's an issue with one of the destructors
• Output data is not affected, the actual code of ShipReco finished running successfully
• I can not reliably reproduce it by deleting/destructing any particular object
• The issue only seems to appear when SND is disabled, but not obvious why. Maybe something to do with the module, data containers?

Help welcome!

[olantwin]

Addendum:

• Behaviour is not deterministic, deleting the same objects in different orders sometimes results in the seg. fault and sometimes does not
• In some cases, the seg. fault does not occur

Nuclear option: rewrite ShipReco, as all the global state makes it extremely annoying to debug

[fairlyons]

I observe a similar segmentation fault when running shipStrawTracking. This also does not occur every time.

[olantwin]

Thanks for the hint! This might allow reducing it to a minimal reproducer, as there's a lot less going on than in ShipReco.