UCL / neso-calibration

Examples of calibrating Neptune Exploratory Software (NESO) models
MIT License
2 stars 0 forks source link

Allow running models asynchronously #4

Closed matt-graham closed 11 months ago

matt-graham commented 1 year ago

In #3 we added the option to use asyncio.subprocess module for dispatching model runs to allow asynchronously redirecting the output streams of the subprocess to the parent process for getting progress updates. At the moment we still expose a synchronous interface for running model however, by passing the coroutine used to set of the model run subprocess to asyncio.run :

https://github.com/UCL/neso-calibration/blob/52e63174f6347d501f7f62771107949be4e299d4/src/nesopy/model.py#L182-L189

It would be nice to instead allow asynchronously dispatching model runs directly from the calling code, to for example allow setting of a batch of runs simultaneously and then returning to the parent process to begin computing the parameters of the next set of runs to dispatch.

We could do this either by making the current model __call__ method an async function instead (that is replacing the current synchronous interface) or by providing an alternative run_async method or similar which can then still be run synchronously in the __call__ method using asyncio.run.