celeritas-project / celeritas

Celeritas is a new Monte Carlo transport code designed to accelerate scientific discovery in high energy physics by improving detector simulation throughput and energy efficiency using GPUs.
https://celeritas-project.github.io/celeritas/user/index.html
Other
62 stars 32 forks source link

Improve device initialization for eventual CMSSW use #1037

Open sethrj opened 9 months ago

sethrj commented 9 months ago

We've never had a completely satisfactory way of handling device initialization:

Important features we want to retain from the current implementation:

It's really unclear to me how in practice we'd want a single process to share multiple GPUs.

drbenmorgan commented 1 month ago

I think the option to enable/disable device use is also required to cleanly implement https://github.com/drbenmorgan/celer-adept/issues/10. I don't know how tightly coupled this is to the other items here, but perhaps we could split it out into a subitem? At least in terms of the celer-adept use case it probably only needs a parameter in SetupOptions and some way to pass that other than an env var into Device?

sethrj commented 1 month ago

Yeah, we can split it into a separate option. For what it's worth, it is possible to set Celeritas "environment variables" programmatically—celeritas::environment().insert({"CELER_DISABLE_DEVICE", "1"}). But there should be a better way than that to disable the device when integrating externally.

The biggest problem is that lots of the code assumes this setting is set exactly once. If we allow it to change then it could result in an inconsistent state. The best option I can think of is to define a more general Config class (perhaps a wrapper to std::map<std::string, std::any>), pass it into the highest level classes (perhaps sharing it with a global config as a default?) and replace static data with class data.