support for more than XYZ axes in acquisition events

[ieivanov]

As I understand it, acquisition events can specify where the images are acquired along the X, Y, and Z axes, defined in Core-XYStage and Core-Focus. Microscopes may have additional axes of motion, for example piezo stages. It would be helpful if these coordinates could be passed to the acquisition events. At least for some stages it's not possible to do that through the 'properties': [['DeviceName', 'PropertyName', 'PropertyValue'] interface as a call to mmc.setPosition() rather than mmc.setProperty is required. Additional stages could be added as

event = {

      #For XY stages
      'x': x_position_in_µm,
      'y': y_position_in_µm,

      #For z stacks
      'z': z_position_in_µm,  # this is the stage 

      #Other stages
      'stage_name': stage_position  # in default units
      }

The acquisition engine would invoke mmc.setPosition(stage_name, stage_position) for axes other than XYZ. Does that make sense?

[henrypinkard]

Yes, and I think this a good idea. But I think this needs to be part of a larger generalization of the acquisition engine, which will also require changes to the core. Essentially, to get rid of the assumptions in the core of a particular microscope setup (i.e. one camera, one XY stage, one Z stage, etc.), and to make this all fully configurable at the acquisition engine level. There's some thoughts about this in this repo, and I have a clearer picture in my head about how this should work that I need to write out at some point

[henrypinkard]

In the mean time, a hacky but workable version can be implemented with acquisition hooks

[ieivanov]

Hi @henrypinkard we still have a need for this, and I wanted to check if we can implement an intermediate solution that will allow us to move an arbitrary stage (e.g. piezo) and a galvo before the planned refactor of the MM core. Do you think this can be easily implemented in the current version of AcqEngJ? I looked around the code a bit, and it didn't seems too difficult, I think, but I'm not sure if there was something I didn't catch.

[henrypinkard]

See here in the "phase 2" section. This type of thing won't require changes to MMCore I think.

I think acquisition hooks are the way to go in the mean time. Any reason you wouldn't be able to use them for this? It seems like it may create more work in the long run to build something specifically for this use case into the acqengj event structure, only to supersede it with a more general mechanism shortly thereafter

[ieivanov]

Thanks Henry, I'll read through the roadmap carefully.

My motivation for not using acquisition hooks was because I would like to run a sequenced acquisition along these additional stages, in which case I would have to define these at the start of the acquisition (correct me if I am wrong).

[henrypinkard]

I think you could do this with hooks. When sequencing is active, it will send a list of acquisition events to the hook rather than just the one acquisition event. So when that happens youd need to create a whole sequence of positions to dispatch to your 2nd stage.

I think you'd have to create the positions of that sequence at the beginning of acquisition, but this isn't any different from how you create your events right?

[ieivanov]

As I understand it, whether sequencing is engaged or not is determined once at the start of the acquisition, is that right? So I need to start an acquisition that uses sequencing and then I can keep adding more events to that sequence. I'll give that a try.

If I start an acquisition that doesn't use sequencing, and then through a hook I had a set of events that could be sequenced, would sequencing be turned on?

Thanks for your help on this.

[henrypinkard]

The it is supposed to work is that every time you call acq.acquire(events), it will sequence over all those events, if possible. So as long as you don't add events one at a time it should work. Lmk if you see some other behavior though

[ieivanov]

Ah, right, I keep forgetting about using multiple calls to acq.acquire(events). Let me try this and I'll let you know how it works. Thanks!

[henrypinkard]

As you point out in https://github.com/micro-manager/pycro-manager/pull/562, that PR actually moves towards adding this functionality. I now think my previous inclination on this was mistaken. It is likely possible to add features like this and move towards a more general AcqEng incrementally and with backwards compatibility, rather than waiting for a big overhaul of the Core/etc (in no small part thanks to all the excellent tests we now have).

561 adds support for multiple Z axes. Right now this is independent of the "z" field in acquisition events, which is something that should probably change. "z" is really shorthand for "the z position of the core-focus" or the stage device if there is only one. There's also not yet sequencing implemented for these multiple z axes, but I don't foresee any special complication with adding it in.

The updated version of this function in #562 provides a place where all acquisition events coming through python are validated, and thus fields can be deprecated and substituted here, allowing for changing the structure of events without breaking backwards compatibility. A good direction would be to use this to start moving towards a more flexible syntax that allows for for more arbitrary control (different numbers of stages, sequencing, and (eventually) cameras).

[ieivanov]

This is great! Being able to define a second z axis in the events dictionary would certainly be convenient. Currently, we are changing the second z axis (the objective z stage, which does not need to be sequences) in acquisition hooks, as we discussed above, and setting our piezo as the Core-Focus stage for z stack acquisitions.

[henrypinkard]

Yeah I imagine thats a fairly common setup. Two complications that arise for enabling this are the axis names and the data. Right, now, the single Z drive PM knows about is given the axis name "z". If there are multiple enabled, then they could have the same names as the device names, but then the axis names alone won't specify that they are focus drives. Maybe there should be additional metadata specifying this.

Also, when displaying data, if you have two z axes and one of them has a full range of values (the piezo), but the other one only has a single value at each xy position, then the normal viewer is going to be very inconvenient, because you'd have to move both a position slider and a slider for your non-piezo z in order to see anything. There might need to be some way to ignore one of the axes (i.e. the second z drive) and just retrieve any image that is present that matches the others.

[henrypinkard]

Also, when displaying data, if you have two z axes and one of them has a full range of values (the piezo), but the other one only has a single value at each xy position, then the normal viewer is going to be very inconvenient, because you'd have to move both a position slider and a slider for your non-piezo z in order to see anything. There might need to be some way to ignore one of the axes (i.e. the second z drive) and just retrieve any image that is present that matches the others.

wait nevermind, I think this actually wouldn't be a problem, since you can specify the axes separately from the hardware instructions. So you just wouldn't have an axis for the second z drive

[ieivanov]

Yes, I think the viewer problem would be solved by

specify the axes separately from the hardware instructions. So you just wouldn't have an axis for the second z drive

Here, having to move the second z-axis (the objective) is an artifact of the sample now being flat, rather than an important dataset parameter. But I think it's still good to keep metadata for that second axis.

I think we should still have a special "z" axis which correspond to the Core-Focus stage and is used when acquiring z stacks. Additional axes should not be named "x", "y", or "z" as these carry special meaning. Or is this too constraining? What is really special about the Core-Focus stage from the perspective of AcqEngJ? Could we acquire a z-stack with a stage that's not linked to Core-Focus?

[henrypinkard]

I think we should still have a special "z" axis which correspond to the Core-Focus stage and is used when acquiring z stacks. Additional axes should not be named "x", "y", or "z" as these carry special meaning. Or is this too constraining?

Yeah that makes sense

What is really special about the Core-Focus stage from the perspective of AcqEngJ? Could we acquire a z-stack with a stage that's not linked to Core-Focus?

Nothing. Its just the default way of figuring out which Z stack to use (other parts of MM do this too)