neurogears / vestibular-vr

Closed-loop VR setup for Rancz Lab
2 stars 0 forks source link

Implement file playback as a trial condition. #54

Open RoboDoig opened 1 month ago

RoboDoig commented 1 month ago

One requirement for the trial logic is to have a 'playback' type block where a previously recorded signal (e.g. flow sensor) is replayed on a trial to drive visual flow / motor motion. This is already implemented in the schema and the playback gain can be set along with flow gain etc.

The actual source of this playback signal, however, was not clear at the time of implementing. As a placeholder, the analog input on H2 is used as a playback source in the Bonsai workflow, but this means that the Hall sensor analog input cannot be used.

Following discussions last week, it was decided to rather use a loaded file as the source of playback. This could be, for example, a saved HARP stream .bin file containing values and timestamps of the flow sensor in a previous trial. This introduces the requirement to load a HARP stream file and replay it in Bonsai in real-time with timing as similar to the original signal as possible. There are a few ways of doing this:

  1. Use software delays in Bonsai between each timestamped sample to try and replicate the signal.
  2. Force the HARP clock on H1 to produce timestamps as fast as possible (e.g. by enabling flow sensor readings) and use those timestamps to implement hardware delays between successive samples in the read .bin file.
  3. Use an external hardware timed source for playback, e.g. another HARP device with analog input capabilities.

The most robust option is probably option 2 which we agreed will be the initial implementation.

Another caveat is that although we can replay e.g. flow sensor signals on a trial - because there is some randomness involved in each trial (random delay between run threshold and halt) the visual feedback in a replay trial is not guaranteed to be the same as on the original trial if the replay signal is played back. If an 'exact' replay is required, we would need to log and reload the real delay times for the trial and override the random delay with recorded values.

RoboDoig commented 1 month ago

N.B. Another option is to only replay visual stimulus on playback trials. This would require logging the frame buffers separately and replaying them as a 'movie'. Also has the challenge of trying to replicate timing on playback trials but might be able to get away with just playing and replaying at a set frame rate.

RoboDoig commented 3 weeks ago

From meeting 6/4/24

Agreed that way forward for playback / open-loop should be based on choosing function generator or matrixreader as playback source.

Means there will be two playback sources, either a function generator with schema defined parameters, or a matrixreader that reads some stored velocity bin file. In addition to just playback gain in the schema therefore, there should be "functionGeneratorGain" and "fileReaderGain".

RoboDoig commented 3 weeks ago

A suggestion from Goncalo, in the fileReader condition we could use the update cycle of the flow sensor as the timing for reading a file.

Simply, each time the optical sensor is read a corresponding value from a file is looked up and sent as the playback signal.