LightSheetManager AcqEngJ backend

[henrypinkard]

@jondaniels @bls337 Working on a PR for an initial version of this right now. The immediate goal would be to recreate the most important features for diSPIM control, but with the more performant/reliable/debuggable backend provided by AcqEngJ.

I've gone through the main function that performs acquisition in the diSPIM plugin, and I want to make sure I understand what its doing correctly:

• If I'm understanding correctly, most of the heavy lifting is done on the Tiger controller. The AcquisitionSettings object is passed to the function prepareControllerForAquisition, which converts these into commands accepted by the Tiger. The acquisition code does not use the sequencing API in the MMCore to, for example, pre-load a sequence of Z-positions. Is that right?
• One thing that confuses the above assessment is that core_.loadStageSequence (i.e. the core's sequencing API) is called in the preparePlanarCorrectionForAcquisition function. What is planar correction? And how important is it to implement in this first pass?
• The number of (software) triggers sent to the controller is dependent on the specific acquisition settings. In some cases, only a single (software) trigger is sent to the controller and it kicks off a multiple timepoint, 3D, (multiple channel?) acquisition, correct?
• In other cases, there is a (software) trigger for each time point?
• Is there a (software) trigger for each channel always, or sometimes?
• There is a "separate timepoints" mode where each timepoint is essentially its own acquisition, with its own viewer and own file saving. What was the original reason for this? It it still needed in the new version in light of @bls337 's playlist feature?
• What is "stage scanning" mode?
• Multiple positions: diSPIM uses the micro-manager position list. Are there more than just a single XY and single Z stage? And it seems like there may be something important here about the order in which they are moved? And these will always be controlled from API calls, not through the tiger?
• Seems like the SPIM head gets lifted up before moving between XY positions and then re-lowered to prevent objective crashing. Is that right?
• Acquisition order: It seems like the acquisition order is hard-coded to be "xy positions--time points--channels--volumes". Correct? Do we want to expose any other orders as options? Maybe timelapses at each position? (or will playlists handle this?)

[jondaniels]

Here is my take, @bls337 may have clarifications or completely different ideas ;-)

• If I'm understanding correctly, most of the heavy lifting is done on the Tiger controller. The AcquisitionSettings object is passed to the function prepareControllerForAquisition, which converts these into commands accepted by the Tiger. The acquisition code does not use the sequencing API in the MMCore to, for example, pre-load a sequence of Z-positions. Is that right? Correct.
• One thing that confuses the above assessment is that core_.loadStageSequence (i.e. the core's sequencing API) is called in the preparePlanarCorrectionForAcquisition function. What is planar correction? And how important is it to implement in this first pass? This feature is an advanced one that can be omitted at first.
• The number of (software) triggers sent to the controller is dependent on the specific acquisition settings. In some cases, only a single (software) trigger is sent to the controller and it kicks off a multiple timepoint, 3D, (multiple channel?) acquisition, correct? Under most circumstances the plugin triggers separate single-volume acquisitions in the controller. If the expected time between finishing one time point/position and starting the next is less than a semi-arbitrary 0.75 second threshold, a special “hardware timepoints” mode is invoked silently. In the "hardware timepoints" mode only a single software trigger is sent and the controller handles the rest itself. Also note that there may be other serial traffic sent besides the software trigger.
• In other cases, there is a (software) trigger for each time point? Yes, see above.
• Is there a (software) trigger for each channel always, or sometimes? It depends on the channel mode. In "interleave" mode all the channels are done with one software trigger. The plugin calculates how many images it is expecting per trigger.
• There is a "separate timepoints" mode where each timepoint is essentially its own acquisition, with its own viewer and own file saving. What was the original reason for this? It it still needed in the new version in light of @bls337 's playlist feature? The initial reason was to have logical file chunks that could be operated on before the overall acquisition was complete. The playlist feature mostly if not completely supplants the need for this mode.
• What is "stage scanning" mode? There are several acquisition modes that use the stage to move the sample through the light sheet instead of moving the light sheet through the sample. These are all "stage scanning" and there are variants like having the 2nd view be the same stage direction as the first or the reverse direction.
• Multiple positions: diSPIM uses the micro-manager position list. Are there more than just a single XY and single Z stage? And it seems like there may be something important here about the order in which they are moved? And these will always be controlled from API calls, not through the tiger? The use cases I can think of just use XY and Z, but maybe there is something I am not anticipating. I think the only reason to care about the motion order involves the SPIM head optionally being lifted between positions.
• Seems like the SPIM head gets lifted up before moving between XY positions and then re-lowered to prevent objective crashing. Is that right? This is an optional behavior which had good reasons for becoming an option. This feature could be omitted initially but eventually we would want to be able to do this. It could be implemented as a runnable between positions I think.
• Acquisition order: It seems like the acquisition order is hard-coded to be "xy positions--time points--channels--volumes". Correct? Do we want to expose any other orders as options? Maybe timelapses at each position? (or will playlists handle this?) I believe this is correct, though there may be some corner case where a different order is possible in the existing plugin. It would be fine to expose other options, but the playlist helps cover a lot of the corner cases.

[henrypinkard]

Thanks! This is really helpful and I think I should able to put together a skeleton of how this should work in AcqEngJ which we can build on from there.

• Is there a (software) trigger for each channel always, or sometimes? It depends on the channel mode. In "interleave" mode all the channels are done with one software trigger.

Can you set a bit more about interleave mode? What exactly is interleaved? Is it performing successive scans of the same volume in different channels, or something different?

[jondaniels]

• Is there a (software) trigger for each channel always, or sometimes? It depends on the channel mode. In "interleave" mode all the channels are done with one software trigger.

Can you set a bit more about interleave mode? What exactly is interleaved? Is it performing successive scans of the same volume in different channels, or something different

Interleaved rotates through the channels within the same timepoint/position. In essence this does change the acquisition order contrary to my former claim.

For example if you acquiring a single volume of 5 images with 2 channels with interleaved, there would only be one motion but the lasers A and B would fire ABABABABAB instead of one motion with AAAAA and then another with BBBBB. The enum is MultiChannelMode which becomes field channelMode of the acqusitionSettings structure. Triggering different lasers sequentially is handled by the Tiger controller; in the above example the Tiger would have been told to acquire 10 images in the volume and the Tiger would have changed the lasers as configured by setupHardwareChannelSwitching() in the ControllerUtils class.

[henrypinkard]

Thanks! I just completed the first steps of this integration in #9

This adds in the high performance saving in NDTiff with optional display in NDViewer, and it does the beginnings of translating the LSM AcquisitionSettings into AcquisitionEvents and AcquisitonHooks that interact with AcqEngJ. This is using the same system as pycro-manager, so the pycro-manager documentation gives an idea of how the system works. The basic idea is shown in the figure below.

The code is full of TODOs. I think it will be helpful to go through it together at our next meeting