Registration pipeline

[edyoshikun]

This PR implements a registration pipeline for phase and fluorescence on mantis.

2022-07-27 tasks pending:

• [x] Write metadata into each position
• [x] Add the input path validation to parsing.py
• [x] (Opened Issue #83 ) Crop the registered image to proper ROI after transformation
• [x] Save the affine_transform_parameters as a yaml instead of zarr`
• [x] (Opened Issue #84 ) Save both registered channels (phase and fluorescence) in the same zarr.
• [x] Slurmkit registration

[edyoshikun]

Here is an output of the registration, which seems to have a bit of a mismatch in the outer z-planes. https://github.com/czbiohub-sf/mantis/assets/10470962/a6005041-06d5-46af-9aa1-238d9864ee5b Data path: /hpc/projects/comp.micro-1/mantis/2023_05_10_PCNA_RAC1/timelapse_1/8-test-3d-manualreg/movie.mp4

[talonchandler]

Thanks for the example, @edyoshikun. I have a few questions:

• (not strictly related to this PR) let's take a look at the 2023_05_10 phase reconstruction together. The nuclei seem to have a contrast inversion (expected from raw data, but not the phase reconstruction)
• your registered.zarr only seems to have a Phase3D channel. Did the fluorescence channel get saved correctly or is this saving/resampling still in progress?
• I tried to inspect your registration_parameters.zarr in napari and I received

  ValueError: Input data should be an array-like object, or a sequence of arrays of decreasing size. Got arrays in incorrect order, shapes: [(1, 1, 1, 4, 4), (1, 1, 1, 5, 3), (1, 1, 1, 5, 3)]

  Have you found a good way of inspecting and/or modifying these variables?

[edyoshikun]

The registered.zarr output at the moment doesn't copy the fluorescence data to the new store. I was debating if that would have to do that or avoid duplicating the data.

For inspecting the registration parameters, I hadn't thought of a way to inspect the values other than parsing them through code. The regsitration_parameters.zarr has the affine transform and the points used for registration.

[edyoshikun]

Edit: moved these tasks to top.

[ieivanov]

Largely a general comment here: since this is a step towards more robust registration, I would suggest keeping the effort on the manual registration component light - this can be rather unpolished. It's valuable to spend more time on setting up the more general registration infrastructure - once we have transformation matrices (which we can get by manual clicking or otherwise) how to we apply them to the data. I think this is already what you are doing here, but I figured I'll make a note anyway

[ieivanov]

@edyoshikun let me know when you'd like to get a review from me

[edyoshikun]

@talonchandler thanks for the merges!! :)

[talonchandler]

Just merged main into this branch and added some simple tests for the registration cli (that failing while we sort out the configs).

@edyoshikun this merge was quite involved, so please excuse accidental regressions. The tests from main are passing, so I don't expect regressions from outside the registration pipeline.

[edyoshikun]

@talonchandler ready for review.

[talonchandler]

I've run through a start-to-finish estimation and application of the registration, and it's working really well. Approving and merging...great work on this @edyoshikun.

On this iteration we've identified a couple metadata-handling gaps that we'll document these in upcoming issues and address shortly. These issues are minor, and @edyoshikun will be able to use the work on this branch to complete all of the registrations on the latest data.