saving state

[jluastro]

I have a couple of questions regarding state handling for especially long runs:

• Is there any way to save the state of the sampler to a file and then reload it and keep going? This could be in between calls to run_nested() or potential some kind of flushing mechanism every N calls or N iterations.
• Is there someway to visualize what is happening in the fitter in real time? I know there is the print_function; but I want to see some plots on slower timescales than I want to see the printing (to make sure things are still running).

I included these in one issue as they are connected in terms of implementation.

Thanks! Jessica Lu

[joshspeagle]

Hey Jessica,

Is there any way to save the state of the sampler to a file and then reload it and keep going? This could be in between calls to run_nested() or potential some kind of flushing mechanism every N calls or N iterations.

If you want to save the state of the sampler, just pickle-ing the whole thing should work.

Is there someway to visualize what is happening in the fitter in real time? I know there is the print_function; but I want to see some plots on slower timescales than I want to see the printing (to make sure things are still running).

dynesty has the ability to be run as a generator; I have an outline in the docs and one of the Jupyter notebook demos. Running as a generator allows you to do things during the sampling such as making intermediate plots, dumping data to disk, etc., which should hopefully be enough to do what you're looking for. One such example of what this might look like is the implementation in prospector.

[joshspeagle]

Closing this.