Current version of UCLCHEM footprint increases when the python wrapper is used in a script to perform multiple models in a row, which is the default scenario for our grids: https://uclchem.github.io/docs/running_a_grid.
It was determined that this was probably due to the interplay of Fortran and f2py. Since we cannot really address it directly, the best solution is to make sure workers are restarted every so often. This can be done using the joblib workers, i.e. with Parallel and delayed:
from joblib import Parallel, delayed
results = Parallel(n_jobs=6)(
delayed(pool_func)(row) for row in model_table.iterrows()
)
Current version of UCLCHEM footprint increases when the python wrapper is used in a script to perform multiple models in a row, which is the default scenario for our grids: https://uclchem.github.io/docs/running_a_grid.
It was determined that this was probably due to the interplay of Fortran and f2py. Since we cannot really address it directly, the best solution is to make sure workers are restarted every so often. This can be done using the joblib workers, i.e. with
Parallel
anddelayed
: