teamtomo / libtilt

Image processing for cryo-electron microscopy in PyTorch
Other
14 stars 5 forks source link

Projection from 2D -> 1D for common lines #67

Closed McHaillet closed 4 months ago

McHaillet commented 4 months ago

@alisterburt I started some work on a PR for the 2D -> 1D projection. I copied and adapted the code your wrote for the real space 3D->2D projection to create an equivalent function. It would be nice if you could give some feedback on that code and where to place it. I now added it to project_real but some other libraries seems to have separate files for 3d/2d, should that be the case for this as well?

The same still needs to be done for Fourier space 2D->1D but that seems more complicated as besides functions in project_fourier, it also requires a sample_dft_2d in interpolation.

I updated the coarse alignment example as well to test if the proper tilt-axis angle could be optimised based on common lines. Seems to work quite nicely! I start with an angle that is way off (-30, while the actual angle is -88.7), and it finds the correct angle back. Importantly, I need to run it on the raw tilt-series. If I first apply the xcorr shifts, the tilt-axis is quite off (-100).

This is a PR that closes #65 (partly).

alisterburt commented 4 months ago

The same still needs to be done for Fourier space 2D->1D but that seems more complicated as besides functions in project_fourier, it also requires a sample_dft_2d in interpolation.

Good point! I can add this

McHaillet commented 4 months ago

I managed to make the optimization work on the coarse aligned tilt-series. Some images here of the 2D -> 1D projections after optimizing the tilt-axis angle. (each row is a projected tilt image)

Firstly this was the set of lines when optimizing on the 'raw' tilt-series: optimized_common_lines_no_alignment

These were the lines when optimizing on the aligned tilt-series: optimized_common_lines_with_alignment

On these I noticed that the right and left sides get some artifacts which come from the fact that the tilt image is masked before projecting. The line is then normalized to mean0std1, which creates artifacts from the mask.

After some experimenting, I thought to do a 2D->1D projection of the mask to correctly weight the information in the lines. This manages to make the axis angle optimization work also after applying the shifts :smile:

optimized_lines_mask_weights

About the wrap-around thing, it seems to work here now. But still likely that it can cause some issues.

McHaillet commented 4 months ago

Thanks!!

I had another thought when rereading the code - the 'grid' with one grid point is useful only if we might later want to increase the resolution to model the tilt axis varying over time... otherwise a bare torch parameter would look more normal

I left it because I tried optimizing with multiple grid points that vary with tilt angle, which is described in the AreTomo paper. Although in my experience running AreTomo it only ever optimizes a single tilt-axis (in-plane rotation). But I agree, varying over time could make sense!