search

leginon-org / leginon-redmine-archive

1 stars 0 forks source link

ctf bug #2004

Open leginonbot opened 5 months ago

leginonbot commented 5 months ago

Author Name: Scott Stagg (Scott Stagg) Original Redmine Issue: 2004, https://emg.nysbc.org/redmine/issues/2004 Original Date: 2012-08-17 Original Assignee: Scott Stagg

There seems to be another bug with the CTF estimation code:

Starting image 14 ( skip:2566, remain:5579 ) id:1467401, file: 12jun20b_grid8-2_043gr_01sq_013hl_02ex
 ... Pixel size: 0.65204
 ... Best CTF run info: runname='acerun2', confidence=0.778, defocus=2.744 um
 ... Best CTF run info: runname='acerun2', confidence=0.778, defocus=-2.744 um
214710.753941 1.4e-05 6.5204e-11
Defocus search range: 2436.0 A to 52436.0 A
 ... running ctf estimation at Fri Aug 17 15:36:48 2012
/panfs/storage.local/imb/stagg/software/rhel61/usr/local/bin/ctffind64.exe
12jun20b_grid8-2_043gr_01sq_013hl_02ex.mrc
12jun20b_grid8-2_043gr_01sq_013hl_02ex-pow.mrc
2.7,120.0,0.07,214710.753941,14.0,2
512,100.0,15.0,2436.0,52436.0,1000.0,100.0
 ... ctf estimation completed in 5.84 sec
nominal=1.9e-06, bestdef=2.7e-06,

def_1=5.8e-08, def_2=2.3e-07, astig_angle=38.7, cross_corr=0.020,

 ... reading MRC: 12jun20b_grid8-2_043gr_01sq_013hl_02ex-pow.mrc size:(512, 512) dtype:float32
 ... writing JPEG: 12jun20b_grid8-2_043gr_01sq_013hl_02ex-pow.jpg
 ==== Committing data to database ==== 
 ... Committing ctf parameters for 12jun20b_grid8-2_043gr_01sq_013hl_02ex to database
 ... Reading image...
Final params: def1: 5.85e-08 | def2: 2.25e-07 | angle: 38.7 | defratio 3.85
!!! WARNING: atypical defocus value 0.1 microns (underfocus is positive)
Traceback (most recent call last):
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/ctfestimate.py", line 398, in <module>
    imgLoop.run()
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/appionLoop2.py", line 77, in run
    self.loopCommitToDatabase(imgdata)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/appionLoop2.py", line 124, in loopCommitToDatabase
    return self.commitToDatabase(imgdata)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/ctfestimate.py", line 310, in commitToDatabase
    ctfinsert.validateAndInsertCTFData(imgdata, self.ctfvalues, self.ctfrun, self.params['rundir'])
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfinsert.py", line 43, in validateAndInsertCTFData
    ctfvalues = runCTFdisplayTools(imgdata, ctfvalues, opimagedir)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfinsert.py", line 71, in runCTFdisplayTools
    ctfdisplaydict = ctfdisplay.makeCtfImages(imgdata, ctfvalues)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfdisplay.py", line 944, in makeCtfImages
    ctfdisplaydict = a.CTFpowerspec(imgdata, ctfdata)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfdisplay.py", line 825, in CTFpowerspec
    self.convertDefociToConvention(ctfdata)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfdisplay.py", line 769, in convertDefociToConvention
    self.initfreq, self.cs, self.volts, self.ampcontrast)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctftools.py", line 164, in defocusRatioToEllipseRatio
    ellipratio = radii1[0]/radii2[0]
IndexError: list index out of range

Not sure what is causing this.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-08-17T18:10:31Z

Do you really have a defocus of 0.05 microns?

It is crashing, because the first node (valley) of the CTF is before the first oscillation of Cs.

Neil

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Scott Stagg (Scott Stagg) Original Date: 2012-08-17T18:14:18Z

No, the defocus is not 0.05 microns. The correct defocus is 2.7. You can see above that Ace1 got the estimate right.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-08-18T23:08:59Z

Hi Scott,

How would you recommend I fix this, reject all images with defocus less than 0.1 microns or insert the data and not make any images.

The previous estimate from ACE2 was 2.7 microns, but the estimate that ACE1 returned was 0.22 x 0.05 (5.8e-08) microns and so it did not cross zero before the Cs component did, so my program is just not setup to calculate the ellipse ratio in this special condition.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Scott Stagg (Scott Stagg) Original Date: 2012-08-18T23:19:41Z

I guess, I would have it insert the data but not make any images. That way, the user can see what the result is.

leginonbot commented 5 months ago

Original Redmine Comment Author Name: Scott Stagg (Scott Stagg) Original Date: 2012-08-20T13:31:18Z

Is this a related bug? By the way, for both cases in this issue, the program that is returning near zero defocus is ctffind not ace1. 

tarting image 21 ( skip:569, remain:4183 ) id:133887, file: 11feb25b_036gr_01sq_022hl_04enh
 ... Pixel size: 0.65204
 ... Best CTF run info: runname='acerun5', confidence=0.793, defocus=0.498 um
 ... Best CTF run info: runname='acerun5', confidence=0.793, defocus=-0.498 um
214710.753941 1.4e-05 6.5204e-11
Defocus search range: 2482.4 A to 7482.4 A
 ... running ctf estimation at Mon Aug 20 12:26:13 2012
/panfs/storage.local/imb/stagg/software/rhel61/usr/local/bin/ctffind64.exe
11feb25b_036gr_01sq_022hl_04enh.mrc
11feb25b_036gr_01sq_022hl_04enh-pow.mrc
2.7,120.0,0.07,214710.753941,14.0,4
512,100.0,10.0,2482.4,7482.4,100.0,100.0
 ... ctf estimation completed in 7.05 sec
nominal=1.1e-06, bestdef=5.0e-07,

def_1=2.5e-07, def_2=6.8e-07, astig_angle=-25.6, cross_corr=0.110,

 ... reading MRC: 11feb25b_036gr_01sq_022hl_04enh-pow.mrc size:(512, 512) dtype:float32
 ... writing JPEG: 11feb25b_036gr_01sq_022hl_04enh-pow.jpg
 ==== Committing data to database ==== 
 ... Committing ctf parameters for 11feb25b_036gr_01sq_022hl_04enh to database
 ... Reading image...
Final params: def1: 2.49e-07 | def2: 6.75e-07 | angle: -25.6 | defratio 2.71
Final params: def1: 2.49e-07 | def2: 6.75e-07 | angle: -25.6 | defratio 2.71
 ... Defocus Astig Percent Diff 46.11 -- 2.490e-07, 6.751e-07
Computing power spectra.........
 ... Compute PSD with fieldsize 2048 and 9 images complete in 12.8 sec
 ... Preform a rotational average and remove spikes...
PART 1: SETUP PARAMETERS AND ELLIPTICAL AVERAGE
 ... Number of available peaks is 9
 ... Determine and subtract noise model
PART 2: BACKGROUND NOISE SUBTRACTION
 ... starting peak extension
Peak Extension Complete in 345.95 usec
Noise Model Complete in 81.49 msec
Noise Model Complete in 69.69 msec
Traceback (most recent call last):
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/ctfestimate.py", line 398, in <module>
    imgLoop.run()
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/appionLoop2.py", line 77, in run
    self.loopCommitToDatabase(imgdata)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/appionLoop2.py", line 124, in loopCommitToDatabase
    return self.commitToDatabase(imgdata)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/ctfestimate.py", line 310, in commitToDatabase
    ctfinsert.validateAndInsertCTFData(imgdata, self.ctfvalues, self.ctfrun, self.params['rundir'])
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfinsert.py", line 43, in validateAndInsertCTFData
    ctfvalues = runCTFdisplayTools(imgdata, ctfvalues, opimagedir)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfinsert.py", line 71, in runCTFdisplayTools
    ctfdisplaydict = ctfdisplay.makeCtfImages(imgdata, ctfvalues)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfdisplay.py", line 944, in makeCtfImages
    ctfdisplaydict = a.CTFpowerspec(imgdata, ctfdata)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfdisplay.py", line 851, in CTFpowerspec
    normpowerspec = self.normalizeCtf(powerspec)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfdisplay.py", line 127, in normalizeCtf
    valleydata[part3start:part3end], "below")
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfnoise.py", line 533, in modelCTFNoise
    filterctfdata, contraintFunction)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfnoise.py", line 402, in getAllInitialParameters
    contraintFunction, cutoffper=cutoffper)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfnoise.py", line 263, in fitTwoSlopeFunction
    xdata[:cutoff], ctfdata[:cutoff], contraintFunction, maxfun)
  File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apCtf/ctfnoise.py", line 180, in fitLinear
    z = numpy.polyfit(xdata, ctfdata, 1)
  File "/usr/lib64/python2.6/site-packages/numpy/lib/polynomial.py", line 538, in polyfit
    raise TypeError, "expected non-empty vector for x"
TypeError: expected non-empty vector for x
leginonbot commented 5 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-08-20T14:03:54Z

Yeah, it is related. I need to put in a check to make sure not all points are excluded by the upper bound of the CTF, because otherwise there are no points left to fit the background noise curve.