ZhuangLab / storm-control

Microscope control software
Other
65 stars 67 forks source link

Hardware Zscan stochastic errors crash Hal after update 748ebb2 #113

Open AlistairBoettiger opened 3 years ago

AlistairBoettiger commented 3 years ago

After merging the latest updates from the rewrite of the NI timing in sc_hadware/nationalInstruments/, PyDAQmx now gives stochastic errors (1/100 to 1/1000 movies), which crash Hal. We have seen these on two of our systems. In a typical sequential labeling/MERFISH type experiment this amounts to a few crashes a day. We only observe these errors when using "Hardware Z-scan". Each system gives a different buffer error:

system 1 "PyDAQmx.DAQmxFunctions.DACUnderflowErrow: DAC conversion attempted before data to be converted was available." the error suggests: "Decrease the output frequency to increase the period between DAC conversions, or reduce the size of your output buffer in order to write data more often. If you are using an external clock, check you signal for the presence of noise or glitches."

system 2 PyDAQmx.DAQmxFunctions.OutputBufSizeTooSmallToStartGenError: Generation cannot be started, because the selected buffer size is too small. The buffer size too small error is accompanied by the message "Increasing buffer size. Conflicting Property. Property: DAQmx_Byf_Output_BufSize: Corresponding Value: 1 Minimum Supported Value 2: Status Code: -200609"

This suggests the buffer size has unexpectedly become a very picky parameter leading to fatal crashes due to 'too small' or 'too large'. We are imaging at 10Hz on both systems, which I believe is relatively slow compared to the clock speed in the DAQ.

Both systems versions of storm-control is up-to-date with ZhuangLab master as of this writing. The sc-hardware update linked to the crash is 748ebb2. System 1 has a NI PCIe-6353 DAQ card. System 2 has a NI USB-6212, BNC DAQ.