Restarting a simulation

[pausz]

@felixfung Is it still possible to restart a simulation?

[felixfung]

Restart mode is never been implemented, in this code or in the original Drysdale code.

I think it should stay this way, for the following reasons:

1. No one has ever used it, nor has anyone inquired or requested it. It has never been properly implemented in the current code, nor in the original Drysdale code.
2. A restart mode requires dumping additional data at the end of a simulation. This dumping, and subsequent reading back of the dumped data, needs to be implemented for each class. This implementation is substantially more complicated than straightforward input/output to/from config and output files, so that development time for all developers, in particular the peripheral developers, will explode.
3. Given the relative technical difficulty of implementing the restart mode on a per class basis, the chance of introducing difficult-to-detect numerical bugs explodes. Given this is in a per class basis, it is difficult to maintain.
4. The dump data can be huge. For example, the entire firing history has to be dumped. This tax in performance overhead defeats the original motivation of cutting simulation time, especially as this tax is incurred even when the restart mode is not subsequently used.
5. From the user's point of view, using the restart mode requires correctly handling the config file, the dump file and output file. Probably, the user will need to specifically tinker with the config file or launching routine to correctly use the restart mode. Any incorrect tinkering will give numerical error, which may be difficult to detect. There may well be an additional layer of potential pitfall in different matlab/python/perl launching scripts, which may be tailored by the individual user. Thus on a per simulation basis, each restart simulation is a potential undetected numerical error.

The current way of performing each simulation anew I think satisfies all users.

On Tue, Aug 25, 2015 at 1:52 AM, Paula notifications@github.com wrote:

@felixfung https://github.com/felixfung Is it still possible to restart a simulation?

— Reply to this email directly or view it on GitHub https://github.com/BrainDynamicsUSYD/neurofield/issues/24.

[RomeshA]

I agree that it's complex, and a really compelling case for implementing it is yet to be made

[pausz]

So the documentation in EEGCODE mentioning a 'restart mode' was about a feature that hadn't been implemented or the functionality hadn't been tested?

If it is the latter, it means that there were certain cases where resuming a simulation was necessary or useful.

The possibility to resume a simulation from a previous state would allow users to derive different simulations (S2a, S2b) from an initial simulation (S1) without having to run two lengthy simulations.

S1 --> S2a --> S2b

Running long simulations to lead the system to a particular state (not necessary fixed point solutions ) and using those results as the initial conditions for subsequent simulations is, if anything desirable, instead of having artificial initial conditions. It means that you actually have realistic brain activity in your history.

1. I don't think that's a good reason to decide against it, because I agree with the fact that,
2. given the current dev resources I don't think the implementation will take a short time.
3. Bugs are caught when you add tests to assess the correctness of an implementation. The difficulty also is given by the lack/little support of object serialization support in C++. So, while I agree that restart mode might prove difficult both to implement and to maintain in this code, that doesn't make it an undesirable feature.
4. With high dimensional (eg, imagine a discretised space with +20K points), I wouldn't be concerned about optimizing storage. Yes, data files will be huge, in the order of a several GB for a few seconds worth of simulation sampled at round about 512Hz. I don't think storage is a problem nowadays. Isn't the code currently keeping the history up to the maximum delay? I guess it only does this for Q.
5. You can't avoid other people making mistakes. If users don't know what they are doing. it's not the devs fault .... Unless there isn't a clear documentation indicating how to use the software. Users should only decided whether they want to get the dump file or not. There are restrictions that apply of course. For instance, parameters that may change the spatiotermporal structure of the model shouldn't be allowed (wave velocity that affects the Courant condition, deltat that affects the size of the history, number of of nodes in the grid, and so on ...),

Thanks for the feedback and discussion though.

[felixfung]

Happy that my input is appreciated. On Aug 26, 2015 2:48 AM, "Paula" notifications@github.com wrote:

So the documentation in EEGCODE mentioning a 'restart mode' was about a feature that hadn't been implemented or the functionality hadn't been tested?

If it is the latter, it means that there were certain cases where resuming a simulation was necessary or useful.

The possibility to resume a simulation from a previous state would allow users to derive different simulations (S2a, S2b) from an initial simulation (S1) without having to run two lengthy simulations.

S1 --> S2a --> S2b

Running long simulations to lead the system to a particular state (not necessary fixed point solutions ) and using those results as the initial conditions for subsequent simulations is, if anything desirable, instead of having artificial initial conditions. It means that you actually have realistic brain activity in your history.

1.

I don't think that's a good reason to decide against it, because I agree with the fact that, 2.

given the current dev resources I don't think the implementation will take a short time. 3.

Bugs are caught when you add tests to assess the correctness of an implementation. The difficulty also is given by the lack/little support of object serialization support in C++. So, while I agree that restart mode might prove difficult both to implement and to maintain in this code, that doesn't make it an undesirable feature. 4.

With high dimensional (eg, imagine a discretised space with +20K points), I wouldn't be concerned about optimizing storage. Yes, data files will be huge, in the order of a several GB for a few seconds worth of simulation sampled at round about 512Hz. I don't think storage is a problem nowadays. Isn't the code currently keeping the history up to the maximum delay? I guess it only does this for Q. 5.

You can't avoid other people making mistakes. If users don't know what they are doing. it's not the devs fault .... Unless there isn't a clear documentation indicating how to use the software. Users should only decided whether they want to get the dump file or not. There are restrictions that apply of course. For instance, parameters that may change the spatiotermporal structure of the model shouldn't be allowed (wave velocity that affects the Courant condition, deltat that affects the size of the history, number of of nodes in the grid, and so on ...),

Thanks for the feedback and discussion though.

— Reply to this email directly or view it on GitHub https://github.com/BrainDynamicsUSYD/neurofield/issues/24#issuecomment-134864821 .

[j-roberts]

Heh interesting. Sorry to bump a very old thread, but in the old eegcode the restart mode WAS implemented -- I used it, including in a published figure.

A simple and common use case is poor-man's numerical continuation, where you want to change a parameter and continue your solution from where you left off, to find bifurcations etc. Storing the history up to the longest delay was not particularly onerous (I was only interested in the spatially-uniform case back then).

Losing this functionality would be a shame.

[pausz]

Yes, that's what I thought. The EEGCODE classes did have restart methods that unfortunately were not ported to Neurofield.

Another use case is long simulations to simulate minutes even hours of simulated data. Dumping history files and resuming the simulation from the last dump file reduces the memory required for individual simulation processes. Also, if a simulation is really long an it takes hours worth of walltime, a restart mode would provide a mechanism to have data checkpoints in the event of a power outage (which happens a lot around summer).