leginon-org / leginon-redmine-archive

1 stars 0 forks source link

prime number decomposition for CTF estimation #2552

Open leginonbot opened 5 months ago

leginonbot commented 5 months ago

Author Name: Dmitry Lyumkis (@LyumkisLab) Original Redmine Issue: 2552, https://emg.nysbc.org/redmine/issues/2552 Original Date: 2013-10-09 Original Assignee: Saikat Chowdhury


I've recently noticed that my CTF estimation runs off of the K2 data run very slow (on the order of 2-20 minutes per image!). This occurs because ACE2 takes extremely long to generate an FFT (although CTFFind3 seems to have less problems). The prime number decomposition for the images off of Tecnai I (boxsize 3712) is [2, 2, 2, 2, 2, 2, 2, 29]. The 29 seems to cause problems. I would like to put in some code to check for prime numbers, and then clip the image prior to CTF estimation. I think we can just use the function getPrevEvenPrime() in pyami to clip. The question is whether to make this universal, for all CTF estimation algorithms, or just ACE2- (and I guess phasor???)-specific. Thoughts?

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Gabriel Lander (@gclander) Original Date: 2013-10-09T12:41:25Z


I've noticed the same extremely sluggish processing of K2 data with ACE2 (~5 minutes per image). I think Anchi already has the code to pad out K2 images for display/binning purposes, it should be pretty straightforward to incorporate this into ACE2, as well as all the post-analysis (noise subtraction, peak extension, etc). We shouldn't do any padding/clipping for CTFfind since it seems to handle these irregular dimensions without a problem (~20 seconds per image), likely since it was developed to work with rectangular film data. Additionally I'm concerned that padding or clipping will affect the code we have in appion to deal with location-specific CTF correction for tilted images.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2013-10-09T14:15:59Z


The good news is that you only need to add 1 pixel on each side to get a good number.

See #2505 for what I did.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2013-10-09T14:27:25Z


It is amazing how the prime number come into this. I wrote some code for ACE2 to do subfield averaging to work better with rectangular images, but I have not had time to test it.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Dmitry Lyumkis (@LyumkisLab) Original Date: 2013-10-15T19:36:47Z


added some code to the pre-processing do the following for pyace2.py

1) if image contains bad prime number, find nearest good image size (dimension larger than original) 2) pad out image to that dimension, using image mean as a constant frame

this makes ACE2 FFT calculation drop from ~10 minutes to 20 seconds. tested on K2 data, and ACE2 seems to estimate well, on par with the other packages (I'm comparing with CTFFind and phasor). Let me know if anyone finds problems.

This is revision 17944 (sorry didn't associate with commit)

Dmitry

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2013-11-08T23:54:30Z


Gabe, you should have noticed by now if this helped.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Gabriel Lander (@gclander) Original Date: 2013-11-09T00:09:55Z


Saikat, how long does ACE2 take to estimate the CTF now? You're doing a lot more data collection than I am these days, so you'd have a better handle on this than me. The ACE2 step should take about 20 seconds per image.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Gabriel Lander (@gclander) Original Date: 2013-11-09T00:21:54Z


On garibaldi using the myami/trunk the ACE2 step still takes ~4 minutes per image. Does this have to be recompiled on garibaldi?