search

neurogears / vestibular-vr

Closed-loop VR setup for Rancz Lab
2 stars 0 forks source link

ONIX Analog I/O workflow crash #37

Closed RoboDoig closed 4 months ago

RoboDoig commented 4 months ago

In the process of trying to add signal rerouting to the insertion workflow (NPX channel routed to analog out on breakout to display on oscilloscope) we have been encountering a consistent crash state.

Current FMC host hardware version 1.4, firmware 0.4.

To replicate, run a workflow with an AnalogIODevice like MinimalOnixDebug.bonsai. On the first run after opening, the workflow will behave as normal. Stop the workflow and restart to encounter the crash with error: Error in recv. Chnl: 3, Timeout: 0, GetOverlappedResult: 170. This crash will not replicate if no input is given to the AnalogIODevice. The crash also doesn't replicate on our ONIX board in London (hardware 1.6, firmware 0.6)

RoboDoig commented 4 months ago

The only other reference I can find similar to the GetOverlappedResult is in a discussion of using oni-repl to avoid the undesired hardware state that can be cause by the current Bonsai libraries. Also possible that closing the workflow is leading to an undesired hardware state which causes the error on restart.

RoboDoig commented 4 months ago

Also interestingly, I can't replicate this in OnixAnalogDebug workflow (where there are just key presses to trigger analog writes).

RoboDoig commented 4 months ago

Cause of the issue seems to be with workflow branches where analog IO operator is directly linked to an immediate source of data. When an NPX channel is routed to an analog output directly via a subject presumably the analog IO device does not have time to initialise in the subscription chain before data is immediately fed to it.

If instead, we create a source for the analog IO device in a separate branch, then multicast to that source with a delay in a separate branch, the issue does not reappear.

This is presumably why the key press test workflow does not produce the error. The full subscription chain can be complete along with analog IO device initialisation, no data is immediately pushed until a key press occurs.