search

mpiraux / mpf

Minimal Performance Framework
0 stars 0 forks source link

Parallel executions of @mpf.run functions #14

Closed mpiraux closed 7 months ago

mpiraux commented 8 months ago

This brings support for the optional parallel argument in mpf.run that will make the registered function asynchronous. Its result and %ex logs will be fetched before executing the next non-parallel registered function.

This can greatly decrease the time to perform experience setups in @mpf.run functions.

The PR required reworking how %ex logs are stored. They are still stored in a global variable but whose name will be unique under the condition of non-repeating role, link and function name. For registered functions that will not overlap (i.e. they do not set the parallel argument consecutively), this is not an issue.

All registered functions, either via @mpf.run or @mpf.init, are not required to accept the mpf_ctx argument as a consequence.

mpiraux commented 8 months ago

There is one intended limitation which is that an experiment run cannot end with parallel functions pending, it must be waited for by a synchronous function. This was to simplify the code base.