Solution to timing

[XX-Yin]

@bruno-f-cruz @jsiegle Hi Bruno,

I tested the shaders timer. The average delay was reduced, but the jitter was still high as shown below.

Josh and I decided to use a digital output (could from toggle digital output of Harp or from NI daq) to perform timing of between ITI and go cue. The logic here is we send a digital output with very precise duration at the start of ITI, and then use the falling edge of the digital output to trigger sound card to give go cue. Meanwhile, we need to detect the falling edge in Bonsai in order to go to the response period of the task to decide if we should give reward based on the animal's choice. In this way, we can probably get a very precise duration between ITI start and go cue start.

What do you think about this? If it's reasonably practicable, how to detect the falling edge in Bonsai? Is it fast enough? I think the logic here is similar to detect licks.

[bruno-f-cruz]

Hey Are you varying the iti duration on every single trial? If so I don’t believe this is a fair benchmark since, as I mentioned , the shaders timer will only be able to give you accurate times as multiples of update frame events. Regarding the second question I am not sure o follow, can you try to expand on it a bit more? Thanks!

On Sat, 20 May 2023 at 00:06, XX-Yin @.***> wrote:

@bruno-f-cruz https://github.com/bruno-f-cruz @jsiegle https://github.com/jsiegle Hi Bruno,

I tested the shaders timer. The average delay was reduced, but the jitter was still high as shown below. Josh and I decided to use a digital output (could from toggle digital output of Harp or from NI daq) to perform timing of between ITI and go cue. The logic here is we send a digital output with very precise duration at the start of ITI, and then use the falling edge of the digital output to trigger sound card to give go cue. Meanwhile, we need to detect the falling edge in Bonsai in order to go to the response period of the task to decide if we should give reward based on the animal's choice. In this way, we can probably get a very precise duration between ITI start and go cue start.

What do you think about this? If it's reasonably practicable, how to detect the falling edge in Bonsai? Is it fast enough? I think the logic here is similar to detect licks.

[image: image] https://user-images.githubusercontent.com/109394934/239652593-a475bdd9-1ef3-4806-b7a6-49063e133656.png

— Reply to this email directly, view it on GitHub https://github.com/AllenNeuralDynamics/dynamic-foraging-task/issues/9, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVZBBZEHHQDZDQOSZFLLSTXG74IRANCNFSM6AAAAAAYIKMG5Q . You are receiving this because you were mentioned.Message ID: @.***>

[jsiegle]

I see...so if the ITIs are always multiples of 16.6 ms (for a 60 Hz refresh rate), then it should be more precise?

[bruno-f-cruz]

Yes, we could think about ways to formalize this a bit better, but the first benchmark I would do is to try the same test but with a constant ITI on every single trial. Just to remove the contribution of some sort of “aliasing” between the two timers.

On Sat, 20 May 2023 at 00:25, Josh Siegle @.***> wrote:

I see...so if the ITIs are always multiples of 16.6 ms (for a 60 Hz refresh rate), then it should be more precise?

— Reply to this email directly, view it on GitHub https://github.com/AllenNeuralDynamics/dynamic-foraging-task/issues/9#issuecomment-1555368955, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVZBB3ZWTQLM3O2MDOB4W3XG76PFANCNFSM6AAAAAAYIKMG5Q . You are receiving this because you were mentioned.Message ID: @.***>

[XX-Yin]

The ITIs are supposed to be different and randomized in each trial. And if the shaders timer can only give you accurate times as multiples of update frame, I do not think it could be applied to our case.

As the toggle digital output is very precise, we probably can use it as a timer. If we want to determine the time interval between ITI start and go cue, we can send a digital output at the start of ITI and then trigger the go cue through the falling phase of the digital output. In this way, the interval between ITI start (determined by the rising phase of the digital output) and go cue start (determined by the falling phase of the digital output) is exactly the same as the digital duration. In this way, we can precisely control the duration between ITI start and go cue start. However, we still need to go back to Bonsai workflow as we need to determine if we should give a reward to this animal during the response period (after a certain period of go cue), so we need to detect the falling phase of that digital output in bonsai code and start triggering the chain after GoCue as shown below.

[bruno-f-cruz]

We have talked before about adding a new event to the set/toggle register that sends an event once the pulse has finished but we never went past that. Currently the best way to achieve this would be to short one of the outputs of the board to one of its inputs. But I agree this could be made much easier in the future when using pulse modes.

On Sat, 20 May 2023 at 01:53, XX-Yin @.***> wrote:

The ITIs are supposed to be different and randomized in each trial. And if the shaders timer can only give you accurate times as multiples of update frame, I do not think it could be applied to our case.

As the toggle digital output is very precise, we probably can use it as a timer. If we want to determine the time interval between ITI start and go cue, we can send a digital output at the start of ITI and then trigger the go cue through the falling phase of the digital output. In this way, the interval between ITI start (determined by the rising phase of the digital output) and go cue start (determined by the falling phase of the digital output) is exactly the same as the digital duration. In this way, we can precisely control the duration between ITI start and go cue start. However, we still need to go back to Bonsai workflow as we need to determine if we should give a reward to this animal during the response period (after a certain period of go cue), so we need to detect the falling phase of that digital output in bonsai code and start triggering the chain after GoCue as shown below.

[image: image] https://user-images.githubusercontent.com/109394934/239659796-3505408a-e9e0-4230-a8b7-7cd86e82ac40.png

— Reply to this email directly, view it on GitHub https://github.com/AllenNeuralDynamics/dynamic-foraging-task/issues/9#issuecomment-1555402904, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVZBB6FHFQCL4WNMC4MPXLXHAI2LANCNFSM6AAAAAAYIKMG5Q . You are receiving this because you were mentioned.Message ID: @.***>

[XX-Yin]

That's good to know. If we short one of the outputs of the board to one of its inputs, it means the input needs to be triggered by the falling phase of digital input.

1). Can these inputs in Harp behavior board be triggered by the falling phase of digital input? 2). How many inputs in Harp behavior board can we use in this case? Do we have other inputs besides 3 from port0-2?

[bruno-f-cruz]

I believe that there are a few other inputs. And some dio pins that can be configured as inputs too. Maybe Filipe would be a better person to tag in this issue. Regarding 1) I am not sure I was able to follow the question. Can you expand a bit? Thanks!

[XX-Yin]

The final purpose is to detect the end of a digital output (assuming it's DO1) in bonsai with minimal delay. The strategy is to short DO1 with one digital input (assuming it's DI0). Then, it means that the DI0 should be able to be triggered by the end of DO1 output. The end of DO1 output could be a rising signal or a falling signal. I know that it's easy to set the end of DO1 as a falling signal in Harp behavior board, so if the DI0 could detect the falling phase of input from DO1, then we can probably detect the end of DO1 in Bonsai. The logic is very similar to detect licks. When there is a licking event, we can detect the licking event in Bonsai with very short delay. The licking event is triggered by the rising phase to a DI port. Here, it is equivalent to that the licking event is triggered by the falling phase. If we can not achieve this, we can then try to set end signal of DO1 as a rising phase in some way.

Recently, we are also trying to test the delay between detecting the lick event and giving the reward? Do you know how long the delay is usually?

[bruno-f-cruz]

If I understand correctly you are just asking if the inputs are sensitive to both rising and falling edge interrupts? If so, the answer is yes.

The delay should be a full trip delay, around 2-4 ms most of the time.

[XX-Yin]

Thanks! Yes, then what's the bonsai code for setting up the input that responds to the falling edge interrupt?

2-4ms is good. Is the delay related to CPU memory load?

[bruno-f-cruz]

You should use the same code as the licking detection but look for values equal 0(falling edge) instead of 1(rising edge).

The round trip delay is given mainly by the time it takes to transmit and buffer commands to the harp board.

[jsiegle]

I think there might be a simpler way to do this. This image shows what I think are the minimum connections needed to implement the desired trial structure:

If the go cue (delivered via the Sound Card) is triggered by the rising edge of one of the NIDAQ outputs, then it will be precisely aligned with the end of the ITI stimulation, or the start of the Go cue stimulation.

If we use one of the behavior board digital outputs to indicate when reward was given (or would have been given, on an incorrect trial), that can serve as the hardware trigger for opto stim coincident with reward.

Is this diagram missing anything critical?

[XX-Yin]

You should use the same code as the licking detection but look for values equal 0(falling edge) instead of 1(rising edge).

The round trip delay is given mainly by the time it takes to transmit and buffer commands to the harp board.

Thanks for explanation.

[XX-Yin]

I think there might be a simpler way to do this. This image shows what I think are the minimum connections needed to implement the desired trial structure:

If the go cue (delivered via the Sound Card) is triggered by the rising edge of one of the NIDAQ outputs, then it will be precisely aligned with the end of the ITI stimulation, or the start of the Go cue stimulation.

If we use one of the behavior board digital outputs to indicate when reward was given (or would have been given, on an incorrect trial), that can serve as the hardware trigger for opto stim coincident with reward.

Is this diagram missing anything critical?

I agree with most of them. There are two points I hope to emphasize.

1) I think the NIDAQ AO1 output should be aligned to the start of ITI as you do not know when the go cue will start. The duration of this NIDAQ AO1 output is the same as ITI duration. This digital output can then be used to trigger the sound card. We can also use toggle digital output in harp instead of NIDAQ AO1.

2) In the previous discussions, I mainly asked Bruno about how to detect the end of a digital output in Bonsai. As we still need to decide when we should start the response period (only licks during the response period can be used to decide the reward) ,and the response period starts at the end of ITI.

[hanhou]

I think a general question would be: how could we get a timer with a good precision using only Bonsai/Harp?

The use case is not limited to generating ITI during photostimulation. For example, in a behavioral-only setup, we may want to have a fixed latency (say 200 ms) between the animal's first lick and the reward delivery. In this case, a ~10 ms jitter from Bonsai is even more pronounce.

Also, we should be able to achieve this without any additional NIDAQ. We have NIDAQ for free on ephys rigs, but not on behavioral-only rigs. So it would be ideal if Bonsai/Harp could do this on its own to save us some NIDAQ boards.

What Xinxin has proposed is something like this

which gives us an effective timer of $duration + 2\Delta t$

As Bruno mentioned, it would be even better if a new event could be automatically generated once the DO pulse has finished. In this case we don't need the DI port.

[bruno-f-cruz]

There is another solution that should give you nearly-"ish" the same results. In short:

1. Send your Write command
2. Wait for the Write echo and save the timestamp (t0)
3. Keep listening to all the events from the board and check when the current message timestamp is greater than t0.

Since the board sends periodic events at, at least, 1kHz (i.e. the ADC read) you should be able to get around 4ms or jitter I would expect.

[XX-Yin]

" listening to all the events ". It seems to be a more general method that do not need to pair a digital output with a digital input.

[XX-Yin]

I tested the following logic: DO1 and DI3 are shorted. Then we can detect the falling phase of DI3 in Bonsai and use it to trigger another digital output. The difference (real ITI-expected ITI; real ITI is the interval between the rising phase of DO1 and DO2) is small ~0.0025s, and the jitter is also very small. I think it's good enough for timing in most cases. Not sure what happened on the first trial, the difference was around 40 ms.

[XX-Yin]

There is another solution that should give you nearly-"ish" the same results. In short:

1. Send your Write command
2. Wait for the Write echo and save the timestamp (t0)
3. Keep listening to all the events from the board and check when the current message timestamp is greater than t0.

Since the board sends periodic events at, at least, 1kHz (i.e. the ADC read) you should be able to get around 4ms or jitter I would expect.

It is also good to try this method. It would be great if you could provide an example on how to do this.