Recorder class re-engineering

[stiber]

Right now, the Recorder classes are implemented specially for each model. However, the fundamental operation of a recorder is as follows:

1. A set of variables is specified for recording. Each variable has a name and dimensions (it may be the case that one of the dimensions is unknown ahead of time, for example, for the list of spike times produced by neurons — we know how many neurons, but we don't know how many spikes each neuron will produce).
2. Each variable has a temporal resolution. Some variables need to be recorded (can update) every time step; some will not update more often than once per epoch.
3. In addition, as set of constants may be specified. These are values that do not change during the simulation, such as neuron types, vertex locations, etc.
4. The recorder has an output format, which is captured by the subclass, currently XML or HDF5.

Can we do something like the ParameterManager, where we register variables with a recorder object, providing whatever information is needed at that time (variable name as a string, variable update interval, variable dimensions)? The recorder would have a method that is periodically called to buffer/output variable contents. Using the above scheme, perhaps we would no longer need to provide a list of vertices to attach recorders to, since recording would be at a variable basis and so could be completely generic (we could record edge variables, for instance).

[stiber]

This umbrella issue needs to be broken down into actionable tasks.

[stiber]

Some questions to answer related to this:

• When does the recorder method that periodically captures data to the output get called?
• Who calls it?
• Right now, how does recorder know which variables to capture?
• For capturing data from arrays, how does the recorder know the start and end indices?
• Does the number of items to capture for arrays change during the simulation?

[stiber]

Here is the start of this process:

[stiber]

Also includes #141

[stiber]

These whiteboard images might be relevant:

[PoojaPal2021]

This is based on the discussion - why do we need recorder class reengineering. The overview of chain of commands

[xiangli6]

It is possible to break down this issue into a sequence of steps.

[xiangli6]

First step: migrate variable registration from the Recorder to other components

[xiangli6]

Based on the plan shown in the picture above, issue 240 can be broken down into a sequence of smaller issues. Here is the breakdown:

Part 1: Reengineering XmlRecorder (First, we work on the XmlRecorder)

1. Issue-452: Transitioning Variable Registration to Enable XML Recorder Recording of All Events for a Single/Random Neuron through EventBuffer here This initial step focuses on migrating the variable registration functionality from the Recorder class to the appropriate variable owner classes. You can find more details about this issue .

2. Issue-478: XmlRecorder records all events for all neurons instead of a single neuron here The goal here is to modify the Xml Recorder class to create a variable table, which will enable recording of events for all neurons.

3. Issue-502: Create a recordable template for EventBuffer class and modify the code in the XmlRecorder here The purpose of this issue is to create a recordable template for the EventBuffer class and modify the code in the XmlRecorder so that it can work with Recordable. This modification is necessary because the EventBuffer stores uint64_t, and if there is a need for the simulator to store other types of data such as double, the code needs to be updated accordingly.

4. Issue-503: Add an interface for recording constant variables such as X, Y location in the XmlRecorder here This issue involves adding an interface to the XmlRecorder so that it can record constant variables, such as X and Y location. These constant variables only need to be recorded once because these value do not change as the simulation runs.

5. Issue-504: Add an additional subclass/concrete class to the recordable and enable it working with the XmlRecorder here The objective of this issue is to add a new subclass/concrete class to the recordable framework so that the XmlRecorder can work with it effectively.

[stiber]

Added umbrella issue #535 for HDF5 recorders.

[xiangli6]

One possible solution if we want to use template in the Recorder

[xiangli6]

currently XMLRecorder UML class diagram