search

teamtomo / fidder

Detect and erase gold fiducials in cryo-EM images
https://teamtomo.org/fidder/
Other
11 stars 4 forks source link

Erase features in 3D from 3D mask #35

Open brisvag opened 2 months ago

brisvag commented 2 months ago

First attempt at using fidder to do feature erasure on a volume instead of a stack of 2D images.

To avoid unnecessary refactoring, for now we just made a secondary code path for 3D which is essentially a copy-paste of the 2D one with minor changes.

estimate_local_mean is now replaced by estimate_local_mean_3d, which uses a badly implemented torch_cubic_spline_grids.b_spline_grids.CubicBSplineGrid3d (@alisterburt help pls :P) in order to interpolate in 3D instead of 2D.

Then, we also added an estimate_background_std_3d which is trying to be a bit smarter about noise (since Z noise is quite different from XY noise in tomos).

Any thoughts/suggestions on this approach?

For now, we don't see an obvious improvement on the erased subvolumes (at least visually), but we're testing some downstream processing to see if it affects the STA.

cc @alessiodacapito

estrozi commented 1 month ago

Hi guys!!! If I understood well the goal, IMHO it would be simpler to adapt/change/retrain membrain-seg to have a "--2d" option so it can segment membranes in 2d (at tilt series level) and then apply fidder. Am I wrong? (of course, easier said than done). It would also be possible to "project back" the membrain-seg results (3D masks) onto the tilt series performing the corresponding transformations backwards. Clearly, when re-projected back in 2d the membranes have very different aspects (intensities) depending on the angle their normal make with the tilt axis but they only need to be erased where the intensities are the strongest. So, 2d projections of the membrain-seg result according to the tilt angles, followed by a peak search on a 2D cross-correlation with the raw images simply to put the mask "back" at the right x-y position, and then fidder can do its magic. No ? Perhaps it would be even better, instead of "erasing" the membranes using fidder, they could be subtracted by using the projected tomogram, similar to the SPA signal subtraction. Cheers!

brisvag commented 1 month ago

How about we part out the erasing code for both 2D + 3D into a cryoeraser package? That way these could live together and fidder could depend on just the 2D bit

Yes, I was thinking the same! I think fidder now is really 2 different things in one and the eraser would be useful as standalone. I also think that in that case we should try to merge the paths between 2D and 3D a bit, all this duplication is fragile. But for now I went for separate because it was quicker and easier to review.

It would also be possible to "project back" the membrain-seg results (3D masks) onto the tilt series performing the corresponding transformations backwards.

Yes, that's exactly what I suggested as well, though it's a significantly harder work so we went for the low-hanging fruit first. One major drawback of working in 3D is that the noise pattern is way different (very much not normal distribution, I suspect...) which makes the masking pretty bad. Though, maybe the mistake I made in the grid is also to blame...

alessiodacapito commented 1 month ago

How about we part out the erasing code for both 2D + 3D into a cryoeraser package? That way these could live together and fidder could depend on just the 2D bit

Yes great idea !

It would also be possible to "project back" the membrain-seg results (3D masks) onto the tilt series performing the corresponding transformations backwards.

I am currently working on it, it's showing good results for now !