teamtomo / libtilt

Image processing for cryo-electron microscopy in PyTorch
Other
14 stars 5 forks source link

Finding shifts between images using correlation #70

Closed McHaillet closed 6 months ago

McHaillet commented 6 months ago

Some simple functionality for finding a shift between two images based around correlate_2d. The code uses torch interpolation to upsample the correlation image and find a more precise maximum with bicubic interpolation.

The function now outputs the shift of image b relative to image a. This means that by applying the inverse of the shift to image b, it is aligned with image a. To me that makes sense but let me know what you think.

McHaillet commented 6 months ago

Would it be possible to extend the implementation so that a stack of images in b can be aligned to a reference a? If you don't want to do this here we can merge and add that later

That sounds doable, I can also make it that both a and b can be stacks.

Hmm, I actually think 'find the shift that you should apply to b to align it with a' feels more intuitive... can you explain why you feel -shift feels right to you?

I was thinking about it in the sense of finding how much b is shifted relative to a.

But thinking about it, for an alignment API, finding the parameters to align b to a are more logical. It does make the function description easier to read, so let's go for that!

Perhaps, I can add to the __init__.py a general remark that functions in the alignment library should follow the convention f(a, b) and return the alignment parameters to align b to a.

McHaillet commented 6 months ago

Stacks is a little less straightforward than I thought. As I now only estimate a subpixel peak if the maximum is not too close to the border, I am not really sure how to handle it for a stack. Simplest way to make the code not too complicated would be to only estimate it if all maxima are far enough away from the edge, but that also does not seem ideal. Alternatively, we leave the stack implementation for a later time.

Another question: so far we only support images - we probably want to support volumes too, what should that API look like overall?

Quite similar at first instance I would say. But as cubic peak interpolation wont work for volumes in pytorch, we perhaps write our own subpixel peak interpolation?

McHaillet commented 6 months ago

Updated the code to now return the shift that aligns b with a.