Implement G4 hits in g4ox

[plexoos]

Please try to see if you can use this simple geometry https://github.com/BNLNPPS/esi-g4ox/pull/48/files

[ggalgoczi]

I am trying to test the hit implementation, however I get segmentation fault running esi-g4ox. Do you have an idea why?

I open

esi-shell -t debug

Then pull esi-g4ox repo, build it. I tried running all GDML, including the one you shared here. The backtrace I get:

Thread 1 "simg4ox" received signal SIGSEGV, Segmentation fault.
0x00007f912bc358f7 in __GI_fseek (fp=0x0, offset=0, whence=2) at ./libio/fseek.c:35
35  ./libio/fseek.c: No such file or directory.
(gdb) backtrace
#0  0x00007f912bc358f7 in __GI_fseek (fp=0x0, offset=0, whence=2)
    at ./libio/fseek.c:35
#1  0x00007f9129250a23 in QRng::Load(long&, char const*) ()
   from /usr/local/opticks/lib/libQUDARap.so
#2  0x00007f9129251f0a in QRng::QRng(unsigned int) ()
   from /usr/local/opticks/lib/libQUDARap.so
#3  0x00007f91291ffd26 in QSim::UploadComponents(SSim const*) ()
   from /usr/local/opticks/lib/libQUDARap.so
#4  0x00007f9129bc455f in CSGOptiX::InitSim(SSim*) ()
   from /usr/local/opticks/lib/libCSGOptiX.so
#5  0x00007f9129bd66da in CSGOptiX::Create(CSGFoundry*) ()
   from /usr/local/opticks/lib/libCSGOptiX.so
#6  0x00007f912cf70c12 in G4CXOpticks::setGeometry_(CSGFoundry*) ()
   from /usr/local/opticks/lib/libG4CX.so
#7  0x00007f912cf72134 in G4CXOpticks::setGeometry(G4VPhysicalVolume const*) ()
   from /usr/local/opticks/lib/libG4CX.so
#8  0x00007f912cf7293e in G4CXOpticks::SetGeometry(G4VPhysicalVolume const*) ()
   from /usr/local/opticks/lib/libG4CX.so
#9  0x000055dbe4f38828 in DetectorConstruction::Construct (this=0x55dbe59a1600)
    at /esi/test-esi-g4ox/src/g4app.h:255
#10 0x00007f912dda8d5e in G4RunManager::InitializeGeometry() ()
   from /opt/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/geant4-11.1.2-djnxepkcbdv5nknlkhjcn2fs7uqkn5zy/lib/libG4run.so
--Type <RET> for more, q to quit, c to continue without paging--
#11 0x00007f912dda8f24 in G4RunManager::Initialize() ()
   from /opt/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/geant4-11.1.2-djnxepkcbdv5nknlkhjcn2fs7uqkn5zy/lib/libG4run.so
#12 0x000055dbe4f15c5d in main (argc=5, argv=0x7ffc25844d28)
    at /esi/test-esi-g4ox/src/simg4ox.cpp:67

[ggalgoczi]

For esi-shell -t cats I do not get this segmentation fault, the code runs.

[plexoos]

Can you provide the exact command?

[ggalgoczi]

./build/src/simg4ox -g esi-g4ox/geom/opticks_raindrop.gdml -m esi-g4ox/run.mac

Did run simtox before that.

[plexoos]

I see an issue, though it might be something else:

SScene::addFrames framespec path [$SScene__initFromTree_addFrames] does not exist 
terminate called after throwing an instance of 'CUDA_Exception'
  what():  CUDA call (cudaFree( (void*)tempBuffer ) ) failed with error: 'unspecified launch failure' (/esi/opticks/sysrap/SOPTIX_Accel.h:172)

Aborted (core dumped)

[ggalgoczi]

I get the same error with the default esi-g4ox code. Did you check if it runs after you installed nsight compute and system? Perhaps these wrecked the links?

[plexoos]

Looks like these lines installing the nsight-compute debugging tool interfere with the rest of nvidia libraries:

https://github.com/BNLNPPS/esi-shell/blob/1e0244e788b7e95640f5185dbd39ea7f9b626575/Dockerfile.debug#L38-L45

I removed the tool from the debug image and rebuilt it. I don't see the above error anymore, so please try to use the same image for your tests and let me know if anything is not as expected.

[ggalgoczi]

Thanks! I could not test that because the new Opticks version is using a different call and simtox fails. Have you seen this?

In file included from /esi/GAGAGA/src/simg4ox.cpp:14:
/esi/GAGAGA/src/g4app.h: In constructor 'G4App::G4App(std::filesystem::__cxx11::path)':
/esi/GAGAGA/src/g4app.h:509:21: error: 'HighLevelCreate' is not a member of 'SEvt'
  509 |         : sev(SEvt::HighLevelCreate(SEvt::ECPU)), det_cons_(new DetectorConstruction(gdml_file)),
      |                     ^~~~~~~~~~~~~~~
gmake[2]: *** [src/CMakeFiles/simg4ox.dir/build.make:76: src/CMakeFiles/simg4ox.dir/simg4ox.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:156: src/CMakeFiles/simg4ox.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

[ggalgoczi]

Ohh, nevermind, I see you just pushed a commit describing the change that is needed! CreateOrReuse_ECPU()

[ggalgoczi]

Now I am getting an other error when running esi-g4ox:

./build/src/simg4ox -g GAGAGA/geom/opticks_raindrop.gdml

simg4ox: /esi/opticks/qudarap/QRng.cc:104: static curandState* QRng::Load(long int&, const char*): Assertion `!failed' failed.
Aborted (core dumped)

@plexoos did you test the building of esi-g4ox with new Opticks version?

[plexoos]

In debug mode many asserts trigger. You need to make sure this directory $HOME/.opticks/rngcache/RNG/ is populated with random seeds. I think this command does it:

QCurandState_SPEC=3:0:0 /usr/local/opticks/lib/QCurandStateTest

~# QCurandState_SPEC=3:0:0 /usr/local/opticks/lib/QCurandStateTest
QCurandStateTest: /usr/local/include/plog/Util.h:594: plog::util::Singleton<T>::Singleton() [with T = plog::Logger<0>]: Assertion `!m_instance' failed.
Aborted (core dumped)

[plexoos]

I should probably revert this change https://github.com/BNLNPPS/esi-opticks/commit/ae968aeef83e5b41e09cbdb26f2dd656f6678fc6 I thought controlling the library log verbosity was convenient from an executable but apparently it should be done in a different way which I don't yet understand

[plexoos]

I rebuilt the debug image with the fix. You can try these commands in the updated image:

QCurandState_SPEC=3:0:0 /usr/local/opticks/lib/QCurandStateTest                                                             
git clone -b pfrich-min https://github.com/BNLNPPS/esi-g4ox.git                                                             
cmake -S esi-g4ox/ -B build                                                                                                 
cmake --build build/                                                                                                        
./build/src/simtox                                                                                                          
./build/src/simg4ox -g esi-g4ox/geom/opticks_raindrop.gdml -m esi-g4ox/vis.mac -i 

[ggalgoczi]

Thank you for the debugging. It works now, I implemented G4Hits and G4SensitiveDetector in the g4hits branch. Created a PR.