search

thouska / spotpy

A Statistical Parameter Optimization Tool
https://spotpy.readthedocs.io/en/latest/
MIT License
248 stars 150 forks source link

SCE: Full parameter list per generation #247

Closed lu-kas closed 4 years ago

lu-kas commented 4 years ago

Dear all,

this may be a naive question: Is it possible to get a full list of all parameter sets (SCE), which will be evaluated in a generation?

Background: We would like to spawn all model evaluations computed during a generation at once. This way we hope to reduce the time to solution for a singe generation by utilising a high number of cores. In our setup we have to run a few thousands simulations, each about a few minutes, per generation.

Goal: Having such list, we would like to process it at once and provide spotpy the full result list for further evaluation.

If this is not possible right now, I would like to evaluate options to implement it, as this feature would make us more productive.

Best and thanks Lukas

thouska commented 4 years ago

Hi Lukas, thank you for your message. However, I am not 100% sure if I understand you plan correctly. Do you mean the parameter-set of the different complexes (ngs)? I guess this should be possible somehow, but is not implemented in spotpy. If you mean the all parameter-sets within a complex that would not be possible, as they are generated depending on the previous run within the complex. Details about the parallelization of the sce-ua algorithm as it is implemented in spotpy can be found here: http://vuir.vu.edu.au/767/1/AParallelShuffled_s48_Muttil_.pdf Hope this helps a bit?

lu-kas commented 4 years ago

Hey Tobias,

thanks for the quick reply -- ok, I understand. Btw, I was interested in the latter, i.e. all or a subset of the parameter within a complex.

I will have a closer look into the algorithm, but I feel, that there will be no further potential for parallelisation.

thouska commented 4 years ago

@lu-kas I am afraid I think so too, there is no further potential to speed up the parallelisation, due to the design of sce-ua. However, if you find potential, feel free to reopen this issue.