Improving robustness

[romainbrette]

Calibration gives poor results. Some ideas to improve.

Improving stage calibration

1. Stage calibration should be separated because it is common to all manipulators. This way we first make sure that the stage is well calibrated.
2. If the camera focal plane is indeed parallel to the stage (XY), then it should be possible to make substantial improvements by using subpixel template matching.
3. We should be able to test the quality manually by moving the stage to the clicked position, e.g. with shift+click.
4. The transformation matrix should have identical columns, because a 1 µm command in the X and Y axes should result in the same pixel displacement. This could be used for quality control and/or for correction (we calculate the average pixel movement, then use this to normalize both axes; this should reduce the error a little bit).
5. A movement is rejected if the pipette is not localized near the expected position. If the same movement is rejected twice, the calibration is rejected.
6. We use several passes. After each pass, we measure the error in target movements (based on template matching). If error is worse, we discard the new calibration.
7. When the stage moves back, check that the image is identical (ie localization at the center). If it is not, then it means that there has been a motor problem at some moment. Therefore, calibration is rejected. Recenter the object.

Improving manipulator calibration

1. Check the photos. Photos should be saved to a folder, not pickled.
2. Quality of the z-stack can be checked by the following algorithm: for each image in the stack, find the closest one (highest correlation). It should differ by +- 1 µm.
3. After the first estimate, check that the image has moved back to the initial state, otherwise reject.
4. Use the pixel size obtained from stage calibration to normalize calibration columns.
5. Use subpixel template matching, and quadratic interpolation for the z-stack.
6. When making a movement with tracking, calculate the error, relative the the amplitude of the movement. If this error increases, reject and move back. Instead of moving back, we may move in the direction that minimizes the expected distance from the initial position.

[romainbrette]

It could be interesting to check the precision of template matching with interpolation on the z axis (eg with random movements).

[romainbrette]

Regarding (1) in manipulator calibration: images are already saved in ./screenshots/.

[romainbrette]

One thing I noticed is that there seems to be some oscillation after a z step. This might be due to settings on the controller. In any case, it might be an issue because the program only sleeps for 0.1 second (maybe this should be a parameter).

I note also that the stack uses fast moves. I propose to switch to relative moves, which by default uses step moves on the L&N. However, currently step moves don't specify the velocity, maybe they should. Also, it would make sense to have step moves with fixed step distance.

[romainbrette]

Regarding (2), one subtlety is M[:,i] is the movement for a 1 µm displacement along the axis, in the camera plane and microscope axis. This means the first two coordinates are in pixels and the last one is in µm, so it is not simple to normalize the matrix. A correct way to do it would be to write for each axis the squared length in µm, as a function of µm/pixel and a correction factor (close to 1). This gives a matrix problem with 4 unknowns, and we may want to minimize, say the correction factors ((c-1)^2).

[romainbrette]

It turns out that making relative moves (steps) instead of absolute moves vastly improves the results.

[romainbrette]

(1), (3), (4) and (6) are implemented. Normalization makes a small improvement (but not very significant). We get an error of ~8 pixels. It would be interested to test whether it depends on SNR, eg by changing exposure or taking several images.

(5) is implicitly implemented, ie we only keep the matrix giving best error. For (7): the stage is recentered after each pass, and then the reference offset vector is calculated so that this position is 0. The matrix is not discarded, but it's not a big deal: an error would show up in the next test.

Currently it uses the previous calibration to make the next movement (with fixed targets in camera frame), and we calculate the error. This is probably ok. A possibility would be to keep in memory all the previous matrices and compare them all.

There were issues with corners (which are black), such that it was necessary to crop the image (by 10%). Possibly some preprocessing (high-pass filtering?) might improve the results.

Overall this seems to work ok, with possibly some room for improvement in template matching:

• (2) Subpixel template matching.
• Template matching on image cropped near expectation (to avoid catastrophic errors).
• Preprocessing (band-pass filtering, edge detection, etc).
• Feature-based template matching (eg looking for the pipette tip).

[romainbrette]

For manipulator calibration, I suggest:

• Repeat the final movement to select the best one.
• Error catching. Keep the best calibration in memory, or simply the previous one. When the pipette is not found, this could be due to the previous calibration: we keep that one.
• Check whether ends are reached (not by checking motor ranges but rather the motor position).
• Make sure we have the coverslip position.