Open leginonbot opened 5 months ago
Original Redmine Comment Author Name: Gabriel Lander (@gclander) Original Date: 2013-10-09T12:41:25Z
I've noticed the same extremely sluggish processing of K2 data with ACE2 (~5 minutes per image). I think Anchi already has the code to pad out K2 images for display/binning purposes, it should be pretty straightforward to incorporate this into ACE2, as well as all the post-analysis (noise subtraction, peak extension, etc). We shouldn't do any padding/clipping for CTFfind since it seems to handle these irregular dimensions without a problem (~20 seconds per image), likely since it was developed to work with rectangular film data. Additionally I'm concerned that padding or clipping will affect the code we have in appion to deal with location-specific CTF correction for tilted images.
Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2013-10-09T14:15:59Z
The good news is that you only need to add 1 pixel on each side to get a good number.
See #2505 for what I did.
Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2013-10-09T14:27:25Z
It is amazing how the prime number come into this. I wrote some code for ACE2 to do subfield averaging to work better with rectangular images, but I have not had time to test it.
Original Redmine Comment Author Name: Dmitry Lyumkis (@LyumkisLab) Original Date: 2013-10-15T19:36:47Z
added some code to the pre-processing do the following for pyace2.py
1) if image contains bad prime number, find nearest good image size (dimension larger than original) 2) pad out image to that dimension, using image mean as a constant frame
this makes ACE2 FFT calculation drop from ~10 minutes to 20 seconds. tested on K2 data, and ACE2 seems to estimate well, on par with the other packages (I'm comparing with CTFFind and phasor). Let me know if anyone finds problems.
This is revision 17944 (sorry didn't associate with commit)
Dmitry
Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2013-11-08T23:54:30Z
Gabe, you should have noticed by now if this helped.
Original Redmine Comment Author Name: Gabriel Lander (@gclander) Original Date: 2013-11-09T00:09:55Z
Saikat, how long does ACE2 take to estimate the CTF now? You're doing a lot more data collection than I am these days, so you'd have a better handle on this than me. The ACE2 step should take about 20 seconds per image.
Original Redmine Comment Author Name: Gabriel Lander (@gclander) Original Date: 2013-11-09T00:21:54Z
On garibaldi using the myami/trunk the ACE2 step still takes ~4 minutes per image. Does this have to be recompiled on garibaldi?
Author Name: Dmitry Lyumkis (@LyumkisLab) Original Redmine Issue: 2552, https://emg.nysbc.org/redmine/issues/2552 Original Date: 2013-10-09 Original Assignee: Saikat Chowdhury
I've recently noticed that my CTF estimation runs off of the K2 data run very slow (on the order of 2-20 minutes per image!). This occurs because ACE2 takes extremely long to generate an FFT (although CTFFind3 seems to have less problems). The prime number decomposition for the images off of Tecnai I (boxsize 3712) is [2, 2, 2, 2, 2, 2, 2, 29]. The 29 seems to cause problems. I would like to put in some code to check for prime numbers, and then clip the image prior to CTF estimation. I think we can just use the function getPrevEvenPrime() in pyami to clip. The question is whether to make this universal, for all CTF estimation algorithms, or just ACE2- (and I guess phasor???)-specific. Thoughts?