checkpoint does not allow for modules to be passed to sampler

[ghost]

Dynesty version 2.0.1

Your question

It appears that due to the checkpointing feature pickling the entire sampler object, it's not possible to pass modules references to the sampler? For example, if the loglikelihood is defined in a separate module.

Attempting to use the checkpoint feature gives in this case TypeError: cannot pickle 'module' object. I believe it's a case of https://stackoverflow.com/a/2790884

Do you have any suggestions on a workaround other than either not using the checkpoint feature or else restructuring the code?

[segasai]

Could you give a reproducible example of the problem ? or at least significantly more details about when the error happens.

I'm reluctant to comment further without seeing more info.

[ghost]

Hi segasai, here is a minimal example using the log-likelihood in the tutorial.

In particular if I have a class calling a module, then it breaks; if not, then it doesn't - compare using class MockData(): which will give the error described above and class MockDataNoCf(): which doesn't.

I don't think I'm structuring my code in a standard way so perhaps I should take this opportunity to alter it such that it's compatible.

test_checkpointing.txt testloglike.txt test_cf_0.txt

[segasai]

Thanks for the example.

Given that's very easy to get around by changing your config file to this:

class Config:
    ndim = 3

and then pass the class instead of the module, I don't think there is any issue here. I think it is quite unconventional to pass around the reference to the module -- you should be passing around the configuration object instead.

[segasai]

I'm closing this. Feel free to reopen if you believe there is still an issue.