Closed shoepfl closed 1 year ago
You are not using the dynesty directly, but using something from pypesto instead. So I can't comment on that.
Also you if you are running multiprocessing you should put your code in
if __name__=='__main__':
do_something()
Yes thats correct, the pypesto guys said that it should just pass the arguments to dynesty so I wanted to ask if multiprocessing is a valid argument for pool at all.
I will ty to use the dunder method
if after putting the code inside if name=='main' and using vanilla dynesty.Sampler() still produces the same issue, please write here.
Putting the code inside if name=='main' did not change anything, what do you mean with vanilla dynesty.Sampler?
I mean if you use just the sampler from dynesty, not the pypesto wrapper around it.
This is not possible as my model is in SBML and therefore cannot be passed to dynesty directly. If you have an example code for me I can test if the error persists also by using dynesty directly. The error message indicates that the error is only dependent on dynesty.
I can already see that the issue is caused by the wrapper, specifically by this internal function definition which is not pickleable:
So the issue needs to be addressed there. Dynesty itself doesn't have the limitation and works perfectly fine with multiprocessing pool.
Okay thanks so this already helps a lot as I was wondering the whole time if dynesty is supposed to work with multiprocessing.
Then I will see how to solve it with the wrapper thanks.
Dynesty version 2.1.0, installed via pip
Hey, I am a littel confused. I tried to use multiprocessing to parallelize dynesty as there are also some examples in the issues here.
I use dynesty via pypesto to compile SBML models, my code is:
However, I also gett a pickle error:
"AttributeError: Can't pickle local object 'DynestySampler.initialize..loglikelihood'"
I already implemented it with the manager of mulitprocessing (https://superfastpython.com/multiprocessing-pool-share-with-workers/) but still get the error.
Is multiprocessing supposed to work with dynesty? If not what are good arguments for pool? OpenMPI is not working for me as I am using it for Systems Biology together with amici which conflicts in many ways with OpenMPI
Thanks for any help in advance