search

celeritas-project / celeritas

Celeritas is a new Monte Carlo transport code designed to accelerate scientific discovery in high energy physics by improving detector simulation throughput and energy efficiency using GPUs.
https://celeritas-project.github.io/celeritas/user/index.html
Other
58 stars 32 forks source link

Celeritas offloading interfaces do not support multiple runs in Geant4 #1273

Open drbenmorgan opened 3 weeks ago

drbenmorgan commented 3 weeks ago

I only spotted this in testing Celeritas in the celer-adept integration exercise. Offloading Geant4 tracks to Celeritas using any of the offload interfaces (Simple/FastSimulation/TrackingManager) results in a fatal exception from celeritas on the second invocation of G4RunManager::BeamOn:

G4WT6 > SimpleOffload.cc:149: [7/36] status: Finalizing Celeritas
SimpleOffload.cc:149: status: Finalizing Celeritas
info: Creating 0 device streams
SharedParams.cc:214: status: Initializing Celeritas shared data

-------- EEEE ------- G4Exception-START -------- EEEE -------
*** G4Exception : celer0001
      issued by : /.../src/corecel/sys/Device.cc:303
celeritas::activate_device may be called only once per application
*** Fatal Exception *** core dump ***
 **** Track information is not available at this moment
 **** Step information is not available at this moment

-------- EEEE -------- G4Exception-END --------- EEEE -------

*** G4Exception: Aborting execution ***

If the device activation is only once per program execution, then can we make it safe to call more than once, e.g. first call wins?

sethrj commented 3 weeks ago

Ah, I guess I didn't know that multiple "beam on" could be run per execution 😅 The intended behavior for SharedParams is to be set up once and torn down once. We don't want to recreate the VecGeom geometry every time we run a beam, for example. If begin/end of run is potentially called multiple times per execution, what's the right way to "initialize on startup" and "finalize on teardown"?

drbenmorgan commented 3 weeks ago

It depends... Between runs, Geant4 is in its "idle" state and so the geometry can completely change, the physics can change in certain aspects like parameters and (de)activation of some models (but it's much more limited). I'll have a look through the examples for the relevant cases and what/how gets triggered to rebuild.

However, would a useful first step be to simply allow multiple runs under the assumption that there are no changes in geometry/physics? The canonical use case is simply changing the particle gun configuration between runs and nothing else.

sethrj commented 3 weeks ago

Huh, I didn't know the physics or geometry could change after the program started... it certainly seems like you can't change anything after closing the run manager which is where my misconception comes from.