search

3dem / relion

Image-processing software for cryo-electron microscopy
https://relion.readthedocs.io/en/latest/
GNU General Public License v2.0
453 stars 201 forks source link

Relion 5.0 fails to reconstruct tomograms after tilt-series alignment #1053

Open shahpnmlab opened 11 months ago

shahpnmlab commented 11 months ago

I have aligned tilt-series using relion's python wrappers for AreTomo and now I wish to generate CTF corrected tomograms. However, ReconstructTomograms fails to run because it is unable to find the rlnDefocusU column in the tiltseries star file. 

in:/path/to/relion/relion/src/metadata_table.h, line 463
ERROR: 
Label rlnDefocusU not present in map-41-A_ts_001
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[65341,1],0]
  Exit code:    1
--------------------------------------------------------------------------

The input star file for one of the tilt-series is as follows -

data_map-33-A2_ts_001

loop_
_rlnMicrographMovieName #1
_rlnTomoTiltMovieFrameCount #2
_rlnTomoNominalStageTiltAngle #3
_rlnTomoNominalTiltAxisAngle #4
_rlnMicrographPreExposure #5
_rlnTomoNominalDefocus #6
_rlnCtfPowerSpectrum #7
_rlnMicrographNameEven #8
_rlnMicrographNameOdd #9
_rlnMicrographName #10
_rlnMicrographMetadata #11
_rlnAccumMotionTotal #12
_rlnAccumMotionEarly #13
_rlnAccumMotionLate #14
_rlnTomoXTilt #15
_rlnTomoYTilt #16
_rlnTomoZRot #17
_rlnTomoXShiftAngst #18
_rlnTomoYShiftAngst #19
frames/ts_00180_-42.0.mrc       6       -42.005200      84.700000       116.640000      -5.000000       MotionCorr/job002/frames/ts_00180_-42_0_PS.mrc  MotionCorr/job002/frames/ts_00180_-42_0_EVN.mrc MotionCorr/job002/frames/ts_00180_-42_0_ODD.mrc MotionCorr/job002/frames/ts_00180_-42_0.mrc     MotionCorr/job002/frames/ts_00180_-42_0.star    3.012995        0.000000        3.012995        0.000000        -42.010000      90.336900       -70.699392      892.607040
frames/ts_00175_-39.0.mrc       6       -39.004000      84.700000       113.400000      -5.000000       MotionCorr/job002/frames/ts_00175_-39_0_PS.mrc  MotionCorr/job002/frames/ts_00175_-39_0_EVN.mrc MotionCorr/job002/frames/ts_00175_-39_0_ODD.mrc MotionCorr/job002/frames/ts_00175_-39_0.mrc     MotionCorr/job002/frames/ts_00175_-39_0.star    5.364340        0.000000        5.364340        0.000000        -39.000000      90.336900       326.090448      632.474568
frames/ts_00160_-36.0.mrc       6       -36.005900      84.700000       103.680000      -5.000000       MotionCorr/job002/frames/ts_00160_-36_0_PS.mrc  MotionCorr/job002/frames/ts_00160_-36_0_EVN.mrc MotionCorr/job002/frames/ts_00160_-36_0_ODD.mrc MotionCorr/job002/frames/ts_00160_-36_0.mrc     MotionCorr/job002/frames/ts_00160_-36_0.star    2.678023        0.000000        2.678023        0.000000        -36.010000      90.336900       -6.372432       382.697784
frames/ts_00155_-33.0.mrc       6       -33.006300      84.700000       100.440000      -5.000000       MotionCorr/job002/frames/ts_00155_-33_0_PS.mrc  MotionCorr/job002/frames/ts_00155_-33_0_EVN.mrc MotionCorr/job002/frames/ts_00155_-33_0_ODD.mrc MotionCorr/job002/frames/ts_00155_-33_0.mrc     MotionCorr/job002/frames/ts_00155_-33_0.star    4.586790        0.000000        4.586790        0.000000        -33.010000      90.336900       322.225776      620.176824
frames/ts_00140_-30.0.mrc       6       -30.006600      84.700000       90.720000       -5.000000       MotionCorr/job002/frames/ts_00140_-30_0_PS.mrc  MotionCorr/job002/frames/ts_00140_-30_0_EVN.mrc MotionCorr/job002/frames/ts_00140_-30_0_ODD.mrc MotionCorr/job002/frames/ts_00140_-30_0.mrc     MotionCorr/job002/frames/ts_00140_-30_0.star    1.446462        0.000000        1.446462        0.000000        -30.010000      90.336900       10.639512       424.997280
frames/ts_00135_-27.0.mrc       6       -27.006500      84.700000       87.480000       -5.000000       MotionCorr/job002/frames/ts_00135_-27_0_PS.mrc  MotionCorr/job002/frames/ts_00135_-27_0_EVN.mrc MotionCorr/job002/frames/ts_00135_-27_0_ODD.mrc MotionCorr/job002/frames/ts_00135_-27_0.mrc     MotionCorr/job002/frames/ts_00135_-27_0.star    3.686294        0.000000        3.686294        0.000000        -27.010000      90.336900       338.652576      812.383992
frames/ts_00120_-24.0.mrc       6       -24.005900      84.700000       77.760000       -5.000000       MotionCorr/job002/frames/ts_00120_-24_0_PS.mrc  MotionCorr/job002/frames/ts_00120_-24_0_EVN.mrc MotionCorr/job002/frames/ts_00120_-24_0_ODD.mrc MotionCorr/job002/frames/ts_00120_-24_0.mrc     MotionCorr/job002/frames/ts_00120_-24_0.star    0.843232        0.000000        0.843232        0.000000        -24.010000      90.336900       51.690960       423.449856
frames/ts_00115_-21.0.mrc       6       -21.005200      84.700000       74.520000       -5.000000       MotionCorr/job002/frames/ts_00115_-21_0_PS.mrc  MotionCorr/job002/frames/ts_00115_-21_0_EVN.mrc MotionCorr/job002/frames/ts_00115_-21_0_ODD.mrc MotionCorr/job002/frames/ts_00115_-21_0.mrc     MotionCorr/job002/frames/ts_00115_-21_0.star    2.714616        0.000000        2.714616        0.000000        -21.010000      90.336900       397.291392      524.631168
frames/ts_00100_-18.0.mrc       6       -18.005600      84.700000       64.800000       -5.000000       MotionCorr/job002/frames/ts_00100_-18_0_PS.mrc  MotionCorr/job002/frames/ts_00100_-18_0_EVN.mrc MotionCorr/job002/frames/ts_00100_-18_0_ODD.mrc MotionCorr/job002/frames/ts_00100_-18_0.mrc     MotionCorr/job002/frames/ts_00100_-18_0.star    1.385673        0.000000        1.385673        0.000000        -18.010000      90.336900       79.709832       976.696704
frames/ts_00095_-15.0.mrc       6       -15.005500      84.700000       61.560000       -5.000000       MotionCorr/job002/frames/ts_00095_-15_0_PS.mrc  MotionCorr/job002/frames/ts_00095_-15_0_EVN.mrc MotionCorr/job002/frames/ts_00095_-15_0_ODD.mrc MotionCorr/job002/frames/ts_00095_-15_0.mrc     MotionCorr/job002/frames/ts_00095_-15_0.star    2.349810        0.000000        2.349810        0.000000        -15.010000      90.336900       306.679608      997.590816
frames/ts_00080_-12.0.mrc       6       -12.004300      84.700000       51.840000       -5.000000       MotionCorr/job002/frames/ts_00080_-12_0_PS.mrc  MotionCorr/job002/frames/ts_00080_-12_0_EVN.mrc MotionCorr/job002/frames/ts_00080_-12_0_ODD.mrc MotionCorr/job002/frames/ts_00080_-12_0.mrc     MotionCorr/job002/frames/ts_00080_-12_0.star    1.207524        0.000000        1.207524        0.000000        -12.000000      90.336900       110.982960      1119.829536
frames/ts_00075_-9.0.mrc        6       -9.004710       84.700000       48.600000       -5.000000       MotionCorr/job002/frames/ts_00075_-9_0_PS.mrc   MotionCorr/job002/frames/ts_00075_-9_0_EVN.mrc  MotionCorr/job002/frames/ts_00075_-9_0_ODD.mrc  MotionCorr/job002/frames/ts_00075_-9_0.mrc      MotionCorr/job002/frames/ts_00075_-9_0.star     2.491558        0.000000        2.491558        0.000000        -9.000000       90.336900       254.778696      1049.890248
frames/ts_00060_-6.0.mrc        6       -6.008080       84.700000       38.880000       -5.000000       MotionCorr/job002/frames/ts_00060_-6_0_PS.mrc   MotionCorr/job002/frames/ts_00060_-6_0_EVN.mrc  MotionCorr/job002/frames/ts_00060_-6_0_ODD.mrc  MotionCorr/job002/frames/ts_00060_-6_0.mrc      MotionCorr/job002/frames/ts_00060_-6_0.star     0.622124        0.000000        0.622124        0.000000        -6.010000       90.336900       200.768544      701.068608
frames/ts_00055_-3.0.mrc        6       -3.008450       84.700000       35.640000       -5.000000       MotionCorr/job002/frames/ts_00055_-3_0_PS.mrc   MotionCorr/job002/frames/ts_00055_-3_0_EVN.mrc  MotionCorr/job002/frames/ts_00055_-3_0_ODD.mrc  MotionCorr/job002/frames/ts_00055_-3_0.mrc      MotionCorr/job002/frames/ts_00055_-3_0.star     1.036964        0.000000        1.036964        0.000000        -3.010000       90.336900       206.561664      1074.266064
frames/ts_00040_-0.0.mrc        6       -0.005320       84.700000       25.920000       -5.000000       MotionCorr/job002/frames/ts_00040_-0_0_PS.mrc   MotionCorr/job002/frames/ts_00040_-0_0_EVN.mrc  MotionCorr/job002/frames/ts_00040_-0_0_ODD.mrc  MotionCorr/job002/frames/ts_00040_-0_0.mrc      MotionCorr/job002/frames/ts_00040_-0_0.star     0.520492        0.000000        0.520492        0.000000        -0.010000       90.336900       197.747568      695.582640
frames/ts_00035_3.0.mrc 6       2.994820        84.700000       22.680000       -5.000000       MotionCorr/job002/frames/ts_00035_3_0_PS.mrc    MotionCorr/job002/frames/ts_00035_3_0_EVN.mrc   MotionCorr/job002/frames/ts_00035_3_0_ODD.mrc   MotionCorr/job002/frames/ts_00035_3_0.mrc       MotionCorr/job002/frames/ts_00035_3_0.star      0.344955        0.000000        0.344955        0.000000        2.990000        90.336900       328.740120      568.157328
frames/ts_00020_6.0.mrc 6       5.994450        84.700000       12.960000       -5.000000       MotionCorr/job002/frames/ts_00020_6_0_PS.mrc    MotionCorr/job002/frames/ts_00020_6_0_EVN.mrc   MotionCorr/job002/frames/ts_00020_6_0_ODD.mrc   MotionCorr/job002/frames/ts_00020_6_0.mrc       MotionCorr/job002/frames/ts_00020_6_0.star      1.375461        0.000000        1.375461        0.000000        5.990000        90.336900       254.417112      600.441336
frames/ts_00015_9.0.mrc 6       8.993580        84.700000       9.720000        -5.000000       MotionCorr/job002/frames/ts_00015_9_0_PS.mrc    MotionCorr/job002/frames/ts_00015_9_0_EVN.mrc   MotionCorr/job002/frames/ts_00015_9_0_ODD.mrc   MotionCorr/job002/frames/ts_00015_9_0.mrc       MotionCorr/job002/frames/ts_00015_9_0.star      1.898463        0.000000        1.898463        0.000000        8.990000        90.336900       362.040840      26.947728
frames/ts_00000_12.0.mrc        6       11.991700       84.700000       0.000000        -5.000000       MotionCorr/job002/frames/ts_00000_12_0_PS.mrc   MotionCorr/job002/frames/ts_00000_12_0_EVN.mrc  MotionCorr/job002/frames/ts_00000_12_0_ODD.mrc  MotionCorr/job002/frames/ts_00000_12_0.mrc      MotionCorr/job002/frames/ts_00000_12_0.star     2.814136        1.777108        1.037028        0.000000        11.990000       90.336900       140.695056      771.363648
frames/ts_00005_15.0.mrc        6       14.993800       84.700000       3.240000        -5.000000       MotionCorr/job002/frames/ts_00005_15_0_PS.mrc   MotionCorr/job002/frames/ts_00005_15_0_EVN.mrc  MotionCorr/job002/frames/ts_00005_15_0_ODD.mrc  MotionCorr/job002/frames/ts_00005_15_0.mrc      MotionCorr/job002/frames/ts_00005_15_0.star     2.551087        0.000000        2.551087        0.000000        14.990000       90.336900       0.000000        0.000000
frames/ts_00010_18.0.mrc        6       17.992500       84.700000       6.480000        -5.000000       MotionCorr/job002/frames/ts_00010_18_0_PS.mrc   MotionCorr/job002/frames/ts_00010_18_0_EVN.mrc  MotionCorr/job002/frames/ts_00010_18_0_ODD.mrc  MotionCorr/job002/frames/ts_00010_18_0.mrc      MotionCorr/job002/frames/ts_00010_18_0.star     3.075137        0.000000        3.075137        0.000000        17.990000       90.336900       119.612376      652.177008
frames/ts_00025_21.0.mrc        6       20.990600       84.700000       16.200000       -5.000000       MotionCorr/job002/frames/ts_00025_21_0_PS.mrc   MotionCorr/job002/frames/ts_00025_21_0_EVN.mrc  MotionCorr/job002/frames/ts_00025_21_0_ODD.mrc  MotionCorr/job002/frames/ts_00025_21_0.mrc      MotionCorr/job002/frames/ts_00025_21_0.star     1.002054        0.000000        1.002054        0.000000        20.990000       90.336900       516.334176      814.294944
frames/ts_00030_24.0.mrc        6       23.991200       84.700000       19.440000       -5.000000       MotionCorr/job002/frames/ts_00030_24_0_PS.mrc   MotionCorr/job002/frames/ts_00030_24_0_EVN.mrc  MotionCorr/job002/frames/ts_00030_24_0_ODD.mrc  MotionCorr/job002/frames/ts_00030_24_0.mrc      MotionCorr/job002/frames/ts_00030_24_0.star     2.328060        0.000000        2.328060        0.000000        23.990000       90.336900       13.403880       598.133808
frames/ts_00045_27.0.mrc        6       26.990900       84.700000       29.160000       -5.000000       MotionCorr/job002/frames/ts_00045_27_0_PS.mrc   MotionCorr/job002/frames/ts_00045_27_0_EVN.mrc  MotionCorr/job002/frames/ts_00045_27_0_ODD.mrc  MotionCorr/job002/frames/ts_00045_27_0.mrc      MotionCorr/job002/frames/ts_00045_27_0.star     1.729675        0.000000        1.729675        0.000000        26.990000       90.336900       379.956744      899.873712
frames/ts_00050_30.0.mrc        6       29.991000       84.700000       32.400000       -5.000000       MotionCorr/job002/frames/ts_00050_30_0_PS.mrc   MotionCorr/job002/frames/ts_00050_30_0_EVN.mrc  MotionCorr/job002/frames/ts_00050_30_0_ODD.mrc  MotionCorr/job002/frames/ts_00050_30_0.mrc      MotionCorr/job002/frames/ts_00050_30_0.star     2.334556        0.000000        2.334556        0.000000        29.990000       90.336900       41.088384       292.731408
frames/ts_00065_33.0.mrc        6       32.990100       84.700000       42.120000       -5.000000       MotionCorr/job002/frames/ts_00065_33_0_PS.mrc   MotionCorr/job002/frames/ts_00065_33_0_EVN.mrc  MotionCorr/job002/frames/ts_00065_33_0_ODD.mrc  MotionCorr/job002/frames/ts_00065_33_0.mrc      MotionCorr/job002/frames/ts_00065_33_0.star     1.947744        0.000000        1.947744        0.000000        32.990000       90.336900       467.530056      96.358248
frames/ts_00070_36.0.mrc        6       35.990300       84.700000       45.360000       -5.000000       MotionCorr/job002/frames/ts_00070_36_0_PS.mrc   MotionCorr/job002/frames/ts_00070_36_0_EVN.mrc  MotionCorr/job002/frames/ts_00070_36_0_ODD.mrc  MotionCorr/job002/frames/ts_00070_36_0.mrc      MotionCorr/job002/frames/ts_00070_36_0.star     2.359789        0.000000        2.359789        0.000000        35.990000       90.336900       43.934400       -73.294632
frames/ts_00085_39.0.mrc        6       38.988900       84.700000       55.080000       -5.000000       MotionCorr/job002/frames/ts_00085_39_0_PS.mrc   MotionCorr/job002/frames/ts_00085_39_0_EVN.mrc  MotionCorr/job002/frames/ts_00085_39_0_ODD.mrc  MotionCorr/job002/frames/ts_00085_39_0.mrc      MotionCorr/job002/frames/ts_00085_39_0.star     2.121234        0.000000        2.121234        0.000000        38.990000       90.336900       446.126616      116.332848
frames/ts_00090_42.0.mrc        6       41.989500       84.700000       58.320000       -5.000000       MotionCorr/job002/frames/ts_00090_42_0_PS.mrc   MotionCorr/job002/frames/ts_00090_42_0_EVN.mrc  MotionCorr/job002/frames/ts_00090_42_0_ODD.mrc  MotionCorr/job002/frames/ts_00090_42_0.mrc      MotionCorr/job002/frames/ts_00090_42_0.star     2.230590        0.000000        2.230590        0.000000        41.990000       90.336900       18.815976       288.164952
frames/ts_00105_45.0.mrc        6       44.989200       84.700000       68.040000       -5.000000       MotionCorr/job002/frames/ts_00105_45_0_PS.mrc   MotionCorr/job002/frames/ts_00105_45_0_EVN.mrc  MotionCorr/job002/frames/ts_00105_45_0_ODD.mrc  MotionCorr/job002/frames/ts_00105_45_0.mrc      MotionCorr/job002/frames/ts_00105_45_0.star     2.792162        0.000000        2.792162        0.000000        44.990000       90.336900       528.458904      353.154816
frames/ts_00110_48.0.mrc        6       47.989300       84.700000       71.280000       -5.000000       MotionCorr/job002/frames/ts_00110_48_0_PS.mrc   MotionCorr/job002/frames/ts_00110_48_0_EVN.mrc  MotionCorr/job002/frames/ts_00110_48_0_ODD.mrc  MotionCorr/job002/frames/ts_00110_48_0.mrc      MotionCorr/job002/frames/ts_00110_48_0.star     2.113075        0.000000        2.113075        0.000000        47.990000       90.336900       10.629792       311.298552
frames/ts_00125_51.0.mrc        6       50.989400       84.700000       81.000000       -5.000000       MotionCorr/job002/frames/ts_00125_51_0_PS.mrc   MotionCorr/job002/frames/ts_00125_51_0_EVN.mrc  MotionCorr/job002/frames/ts_00125_51_0_ODD.mrc  MotionCorr/job002/frames/ts_00125_51_0.mrc      MotionCorr/job002/frames/ts_00125_51_0.star     3.029232        0.000000        3.029232        0.000000        50.990000       90.336900       496.969992      651.273048
frames/ts_00130_54.0.mrc        6       53.990100       84.700000       84.240000       -5.000000       MotionCorr/job002/frames/ts_00130_54_0_PS.mrc   MotionCorr/job002/frames/ts_00130_54_0_EVN.mrc  MotionCorr/job002/frames/ts_00130_54_0_ODD.mrc  MotionCorr/job002/frames/ts_00130_54_0.mrc      MotionCorr/job002/frames/ts_00130_54_0.star     1.623457        0.000000        1.623457        0.000000        53.990000       90.336900       20.188440       768.982248
frames/ts_00145_57.0.mrc        6       56.990200       84.700000       93.960000       -5.000000       MotionCorr/job002/frames/ts_00145_57_0_PS.mrc   MotionCorr/job002/frames/ts_00145_57_0_EVN.mrc  MotionCorr/job002/frames/ts_00145_57_0_ODD.mrc  MotionCorr/job002/frames/ts_00145_57_0.mrc      MotionCorr/job002/frames/ts_00145_57_0.star     3.052850        0.000000        3.052850        0.000000        56.990000       90.336900       451.688400      717.711192
frames/ts_00150_60.0.mrc        6       59.991300       84.700000       97.200000       -5.000000       MotionCorr/job002/frames/ts_00150_60_0_PS.mrc   MotionCorr/job002/frames/ts_00150_60_0_EVN.mrc  MotionCorr/job002/frames/ts_00150_60_0_ODD.mrc  MotionCorr/job002/frames/ts_00150_60_0.mrc      MotionCorr/job002/frames/ts_00150_60_0.star     1.470819        0.000000        1.470819        0.000000        59.990000       90.336900       135.057456      570.997512
frames/ts_00165_63.0.mrc        6       62.990500       84.700000       106.920000      -5.000000       MotionCorr/job002/frames/ts_00165_63_0_PS.mrc   MotionCorr/job002/frames/ts_00165_63_0_EVN.mrc  MotionCorr/job002/frames/ts_00165_63_0_ODD.mrc  MotionCorr/job002/frames/ts_00165_63_0.mrc      MotionCorr/job002/frames/ts_00165_63_0.star     3.133337        0.000000        3.133337        0.000000        62.990000       90.336900       431.371656      -162.689472
frames/ts_00170_66.0.mrc        6       65.991100       84.700000       110.160000      -5.000000       MotionCorr/job002/frames/ts_00170_66_0_PS.mrc   MotionCorr/job002/frames/ts_00170_66_0_EVN.mrc  MotionCorr/job002/frames/ts_00170_66_0_ODD.mrc  MotionCorr/job002/frames/ts_00170_66_0.mrc      MotionCorr/job002/frames/ts_00170_66_0.star     1.169989        0.000000        1.169989        0.000000        65.990000       90.336900       233.437464      375.574968

I then modified the starfiles the DefocusU and DefocusV columns from the starfile generated after CtfFind. See below, for contents of modified star file -

# version 30001

data_map-33-A2_ts_001

loop_
_rlnMicrographMovieName #1
_rlnTomoTiltMovieFrameCount #2
_rlnTomoNominalStageTiltAngle #3
_rlnTomoNominalTiltAxisAngle #4
_rlnMicrographPreExposure #5
_rlnTomoNominalDefocus #6
_rlnCtfPowerSpectrum #7
_rlnMicrographNameEven #8
_rlnMicrographNameOdd #9
_rlnMicrographName #10
_rlnMicrographMetadata #11
_rlnAccumMotionTotal #12
_rlnAccumMotionEarly #13
_rlnAccumMotionLate #14
_rlnTomoXTilt #15
_rlnTomoYTilt #16
_rlnTomoZRot #17
_rlnTomoXShiftAngst #18
_rlnTomoYShiftAngst #19
_rlnDefocusU #20
_rlnDefocusV #21
frames/ts_00180_-42.0.mrc       6       -42.0052        84.7000         116.6400        -5.0000         MotionCorr/job002/frames/ts_00180_-42_0_PS.mrc  MotionCorr/job002/frames/ts_00180_-42_0_EVN.mrc MotionCorr/job002/frames/ts_00180_-42_0_ODD.mrc MotionCorr/job002/frames/ts_00180_-42_0.mrc     MotionCorr/job002/frames/ts_00180_-42_0.star    3.0130  0.0000  3.0130  0.0000  -42.0100        90.3369 -70.6994        892.6070        28683.3066      28302.7871
frames/ts_00175_-39.0.mrc       6       -39.0040        84.7000         113.4000        -5.0000         MotionCorr/job002/frames/ts_00175_-39_0_PS.mrc  MotionCorr/job002/frames/ts_00175_-39_0_EVN.mrc MotionCorr/job002/frames/ts_00175_-39_0_ODD.mrc MotionCorr/job002/frames/ts_00175_-39_0.mrc     MotionCorr/job002/frames/ts_00175_-39_0.star    5.3643  0.0000  5.3643  0.0000  -39.0000        90.3369 326.0904        632.4746        31099.5703      30179.7188
frames/ts_00160_-36.0.mrc       6       -36.0059        84.7000         103.6800        -5.0000         MotionCorr/job002/frames/ts_00160_-36_0_PS.mrc  MotionCorr/job002/frames/ts_00160_-36_0_EVN.mrc MotionCorr/job002/frames/ts_00160_-36_0_ODD.mrc MotionCorr/job002/frames/ts_00160_-36_0.mrc     MotionCorr/job002/frames/ts_00160_-36_0.star    2.6780  0.0000  2.6780  0.0000  -36.0100        90.3369 -6.3724 382.6978        29466.4844      29034.4727
frames/ts_00155_-33.0.mrc       6       -33.0063        84.7000         100.4400        -5.0000         MotionCorr/job002/frames/ts_00155_-33_0_PS.mrc  MotionCorr/job002/frames/ts_00155_-33_0_EVN.mrc MotionCorr/job002/frames/ts_00155_-33_0_ODD.mrc MotionCorr/job002/frames/ts_00155_-33_0.mrc     MotionCorr/job002/frames/ts_00155_-33_0.star    4.5868  0.0000  4.5868  0.0000  -33.0100        90.3369 322.2258        620.1768        29260.9590      29142.6133
frames/ts_00140_-30.0.mrc       6       -30.0066        84.7000         90.7200         -5.0000         MotionCorr/job002/frames/ts_00140_-30_0_PS.mrc  MotionCorr/job002/frames/ts_00140_-30_0_EVN.mrc MotionCorr/job002/frames/ts_00140_-30_0_ODD.mrc MotionCorr/job002/frames/ts_00140_-30_0.mrc     MotionCorr/job002/frames/ts_00140_-30_0.star    1.4465  0.0000  1.4465  0.0000  -30.0100        90.3369 10.6395 424.9973        29082.2539      28744.6641
frames/ts_00135_-27.0.mrc       6       -27.0065        84.7000         87.4800         -5.0000         MotionCorr/job002/frames/ts_00135_-27_0_PS.mrc  MotionCorr/job002/frames/ts_00135_-27_0_EVN.mrc MotionCorr/job002/frames/ts_00135_-27_0_ODD.mrc MotionCorr/job002/frames/ts_00135_-27_0.mrc     MotionCorr/job002/frames/ts_00135_-27_0.star    3.6863  0.0000  3.6863  0.0000  -27.0100        90.3369 338.6526        812.3840        30346.2617      29523.1758
frames/ts_00120_-24.0.mrc       6       -24.0059        84.7000         77.7600         -5.0000         MotionCorr/job002/frames/ts_00120_-24_0_PS.mrc  MotionCorr/job002/frames/ts_00120_-24_0_EVN.mrc MotionCorr/job002/frames/ts_00120_-24_0_ODD.mrc MotionCorr/job002/frames/ts_00120_-24_0.mrc     MotionCorr/job002/frames/ts_00120_-24_0.star    0.8432  0.0000  0.8432  0.0000  -24.0100        90.3369 51.6910 423.4499        30000.9297      29631.4375
frames/ts_00115_-21.0.mrc       6       -21.0052        84.7000         74.5200         -5.0000         MotionCorr/job002/frames/ts_00115_-21_0_PS.mrc  MotionCorr/job002/frames/ts_00115_-21_0_EVN.mrc MotionCorr/job002/frames/ts_00115_-21_0_ODD.mrc MotionCorr/job002/frames/ts_00115_-21_0.mrc     MotionCorr/job002/frames/ts_00115_-21_0.star    2.7146  0.0000  2.7146  0.0000  -21.0100        90.3369 397.2914        524.6312        29218.4062      28534.8633
frames/ts_00100_-18.0.mrc       6       -18.0056        84.7000         64.8000         -5.0000         MotionCorr/job002/frames/ts_00100_-18_0_PS.mrc  MotionCorr/job002/frames/ts_00100_-18_0_EVN.mrc MotionCorr/job002/frames/ts_00100_-18_0_ODD.mrc MotionCorr/job002/frames/ts_00100_-18_0.mrc     MotionCorr/job002/frames/ts_00100_-18_0.star    1.3857  0.0000  1.3857  0.0000  -18.0100        90.3369 79.7098 976.6967        29505.5684      28701.0273
frames/ts_00095_-15.0.mrc       6       -15.0055        84.7000         61.5600         -5.0000         MotionCorr/job002/frames/ts_00095_-15_0_PS.mrc  MotionCorr/job002/frames/ts_00095_-15_0_EVN.mrc MotionCorr/job002/frames/ts_00095_-15_0_ODD.mrc MotionCorr/job002/frames/ts_00095_-15_0.mrc     MotionCorr/job002/frames/ts_00095_-15_0.star    2.3498  0.0000  2.3498  0.0000  -15.0100        90.3369 306.6796        997.5908        30087.9746      29861.4414
frames/ts_00080_-12.0.mrc       6       -12.0043        84.7000         51.8400         -5.0000         MotionCorr/job002/frames/ts_00080_-12_0_PS.mrc  MotionCorr/job002/frames/ts_00080_-12_0_EVN.mrc MotionCorr/job002/frames/ts_00080_-12_0_ODD.mrc MotionCorr/job002/frames/ts_00080_-12_0.mrc     MotionCorr/job002/frames/ts_00080_-12_0.star    1.2075  0.0000  1.2075  0.0000  -12.0000        90.3369 110.9830        1119.8295       31103.9355      29688.6250
frames/ts_00075_-9.0.mrc        6       -9.0047         84.7000         48.6000         -5.0000         MotionCorr/job002/frames/ts_00075_-9_0_PS.mrc   MotionCorr/job002/frames/ts_00075_-9_0_EVN.mrc  MotionCorr/job002/frames/ts_00075_-9_0_ODD.mrc  MotionCorr/job002/frames/ts_00075_-9_0.mrc      MotionCorr/job002/frames/ts_00075_-9_0.star     2.4916  0.0000  2.4916  0.0000  -9.0000 90.3369 254.7787        1049.8902       5552.0615       5485.9155
frames/ts_00060_-6.0.mrc        6       -6.0081         84.7000         38.8800         -5.0000         MotionCorr/job002/frames/ts_00060_-6_0_PS.mrc   MotionCorr/job002/frames/ts_00060_-6_0_EVN.mrc  MotionCorr/job002/frames/ts_00060_-6_0_ODD.mrc  MotionCorr/job002/frames/ts_00060_-6_0.mrc      MotionCorr/job002/frames/ts_00060_-6_0.star     0.6221  0.0000  0.6221  0.0000  -6.0100 90.3369 200.7685        701.0686        28589.5176      28049.1016
frames/ts_00055_-3.0.mrc        6       -3.0084         84.7000         35.6400         -5.0000         MotionCorr/job002/frames/ts_00055_-3_0_PS.mrc   MotionCorr/job002/frames/ts_00055_-3_0_EVN.mrc  MotionCorr/job002/frames/ts_00055_-3_0_ODD.mrc  MotionCorr/job002/frames/ts_00055_-3_0.mrc      MotionCorr/job002/frames/ts_00055_-3_0.star     1.0370  0.0000  1.0370  0.0000  -3.0100 90.3369 206.5617        1074.2661       30295.8633      29391.9082
frames/ts_00040_-0.0.mrc        6       -0.0053         84.7000         25.9200         -5.0000         MotionCorr/job002/frames/ts_00040_-0_0_PS.mrc   MotionCorr/job002/frames/ts_00040_-0_0_EVN.mrc  MotionCorr/job002/frames/ts_00040_-0_0_ODD.mrc  MotionCorr/job002/frames/ts_00040_-0_0.mrc      MotionCorr/job002/frames/ts_00040_-0_0.star     0.5205  0.0000  0.5205  0.0000  -0.0100 90.3369 197.7476        695.5826        31101.9316      30812.9941
frames/ts_00035_3.0.mrc         6       2.9948          84.7000         22.6800         -5.0000         MotionCorr/job002/frames/ts_00035_3_0_PS.mrc    MotionCorr/job002/frames/ts_00035_3_0_EVN.mrc   MotionCorr/job002/frames/ts_00035_3_0_ODD.mrc   MotionCorr/job002/frames/ts_00035_3_0.mrc       MotionCorr/job002/frames/ts_00035_3_0.star      0.3450  0.0000  0.3450  0.0000  2.9900  90.3369 328.7401        568.1573        30172.6328      29940.1660
frames/ts_00020_6.0.mrc         6       5.9944          84.7000         12.9600         -5.0000         MotionCorr/job002/frames/ts_00020_6_0_PS.mrc    MotionCorr/job002/frames/ts_00020_6_0_EVN.mrc   MotionCorr/job002/frames/ts_00020_6_0_ODD.mrc   MotionCorr/job002/frames/ts_00020_6_0.mrc       MotionCorr/job002/frames/ts_00020_6_0.star      1.3755  0.0000  1.3755  0.0000  5.9900  90.3369 254.4171        600.4413        29677.2715      29361.1953
frames/ts_00015_9.0.mrc         6       8.9936          84.7000         9.7200          -5.0000         MotionCorr/job002/frames/ts_00015_9_0_PS.mrc    MotionCorr/job002/frames/ts_00015_9_0_EVN.mrc   MotionCorr/job002/frames/ts_00015_9_0_ODD.mrc   MotionCorr/job002/frames/ts_00015_9_0.mrc       MotionCorr/job002/frames/ts_00015_9_0.star      1.8985  0.0000  1.8985  0.0000  8.9900  90.3369 362.0408        26.9477 31643.6699      31445.8984
frames/ts_00000_12.0.mrc        6       11.9917         84.7000         0.0000          -5.0000         MotionCorr/job002/frames/ts_00000_12_0_PS.mrc   MotionCorr/job002/frames/ts_00000_12_0_EVN.mrc  MotionCorr/job002/frames/ts_00000_12_0_ODD.mrc  MotionCorr/job002/frames/ts_00000_12_0.mrc      MotionCorr/job002/frames/ts_00000_12_0.star     2.8141  1.7771  1.0370  0.0000  11.9900 90.3369 140.6951        771.3636        8045.8545       7892.2671
frames/ts_00005_15.0.mrc        6       14.9938         84.7000         3.2400          -5.0000         MotionCorr/job002/frames/ts_00005_15_0_PS.mrc   MotionCorr/job002/frames/ts_00005_15_0_EVN.mrc  MotionCorr/job002/frames/ts_00005_15_0_ODD.mrc  MotionCorr/job002/frames/ts_00005_15_0.mrc      MotionCorr/job002/frames/ts_00005_15_0.star     2.5511  0.0000  2.5511  0.0000  14.9900 90.3369 0.0000  0.0000  30025.6680      29477.5234
frames/ts_00010_18.0.mrc        6       17.9925         84.7000         6.4800          -5.0000         MotionCorr/job002/frames/ts_00010_18_0_PS.mrc   MotionCorr/job002/frames/ts_00010_18_0_EVN.mrc  MotionCorr/job002/frames/ts_00010_18_0_ODD.mrc  MotionCorr/job002/frames/ts_00010_18_0.mrc      MotionCorr/job002/frames/ts_00010_18_0.star     3.0751  0.0000  3.0751  0.0000  17.9900 90.3369 119.6124        652.1770        29814.6074      29597.5938
frames/ts_00025_21.0.mrc        6       20.9906         84.7000         16.2000         -5.0000         MotionCorr/job002/frames/ts_00025_21_0_PS.mrc   MotionCorr/job002/frames/ts_00025_21_0_EVN.mrc  MotionCorr/job002/frames/ts_00025_21_0_ODD.mrc  MotionCorr/job002/frames/ts_00025_21_0.mrc      MotionCorr/job002/frames/ts_00025_21_0.star     1.0021  0.0000  1.0021  0.0000  20.9900 90.3369 516.3342        814.2949        31097.4121      30590.8984
frames/ts_00030_24.0.mrc        6       23.9912         84.7000         19.4400         -5.0000         MotionCorr/job002/frames/ts_00030_24_0_PS.mrc   MotionCorr/job002/frames/ts_00030_24_0_EVN.mrc  MotionCorr/job002/frames/ts_00030_24_0_ODD.mrc  MotionCorr/job002/frames/ts_00030_24_0.mrc      MotionCorr/job002/frames/ts_00030_24_0.star     2.3281  0.0000  2.3281  0.0000  23.9900 90.3369 13.4039 598.1338        6847.0420       6699.9614
frames/ts_00045_27.0.mrc        6       26.9909         84.7000         29.1600         -5.0000         MotionCorr/job002/frames/ts_00045_27_0_PS.mrc   MotionCorr/job002/frames/ts_00045_27_0_EVN.mrc  MotionCorr/job002/frames/ts_00045_27_0_ODD.mrc  MotionCorr/job002/frames/ts_00045_27_0.mrc      MotionCorr/job002/frames/ts_00045_27_0.star     1.7297  0.0000  1.7297  0.0000  26.9900 90.3369 379.9567        899.8737        5539.6914       5309.2119
frames/ts_00050_30.0.mrc        6       29.9910         84.7000         32.4000         -5.0000         MotionCorr/job002/frames/ts_00050_30_0_PS.mrc   MotionCorr/job002/frames/ts_00050_30_0_EVN.mrc  MotionCorr/job002/frames/ts_00050_30_0_ODD.mrc  MotionCorr/job002/frames/ts_00050_30_0.mrc      MotionCorr/job002/frames/ts_00050_30_0.star     2.3346  0.0000  2.3346  0.0000  29.9900 90.3369 41.0884 292.7314        29689.9805      29052.2539
frames/ts_00065_33.0.mrc        6       32.9901         84.7000         42.1200         -5.0000         MotionCorr/job002/frames/ts_00065_33_0_PS.mrc   MotionCorr/job002/frames/ts_00065_33_0_EVN.mrc  MotionCorr/job002/frames/ts_00065_33_0_ODD.mrc  MotionCorr/job002/frames/ts_00065_33_0.mrc      MotionCorr/job002/frames/ts_00065_33_0.star     1.9477  0.0000  1.9477  0.0000  32.9900 90.3369 467.5301        96.3582 5375.1499       5296.6030
frames/ts_00070_36.0.mrc        6       35.9903         84.7000         45.3600         -5.0000         MotionCorr/job002/frames/ts_00070_36_0_PS.mrc   MotionCorr/job002/frames/ts_00070_36_0_EVN.mrc  MotionCorr/job002/frames/ts_00070_36_0_ODD.mrc  MotionCorr/job002/frames/ts_00070_36_0.mrc      MotionCorr/job002/frames/ts_00070_36_0.star     2.3598  0.0000  2.3598  0.0000  35.9900 90.3369 43.9344 -73.2946        32677.8320      32058.1699
frames/ts_00085_39.0.mrc        6       38.9889         84.7000         55.0800         -5.0000         MotionCorr/job002/frames/ts_00085_39_0_PS.mrc   MotionCorr/job002/frames/ts_00085_39_0_EVN.mrc  MotionCorr/job002/frames/ts_00085_39_0_ODD.mrc  MotionCorr/job002/frames/ts_00085_39_0.mrc      MotionCorr/job002/frames/ts_00085_39_0.star     2.1212  0.0000  2.1212  0.0000  38.9900 90.3369 446.1266        116.3328        5557.0405       5516.1826
frames/ts_00090_42.0.mrc        6       41.9895         84.7000         58.3200         -5.0000         MotionCorr/job002/frames/ts_00090_42_0_PS.mrc   MotionCorr/job002/frames/ts_00090_42_0_EVN.mrc  MotionCorr/job002/frames/ts_00090_42_0_ODD.mrc  MotionCorr/job002/frames/ts_00090_42_0.mrc      MotionCorr/job002/frames/ts_00090_42_0.star     2.2306  0.0000  2.2306  0.0000  41.9900 90.3369 18.8160 288.1650        5631.2534       5405.4985
frames/ts_00105_45.0.mrc        6       44.9892         84.7000         68.0400         -5.0000         MotionCorr/job002/frames/ts_00105_45_0_PS.mrc   MotionCorr/job002/frames/ts_00105_45_0_EVN.mrc  MotionCorr/job002/frames/ts_00105_45_0_ODD.mrc  MotionCorr/job002/frames/ts_00105_45_0.mrc      MotionCorr/job002/frames/ts_00105_45_0.star     2.7922  0.0000  2.7922  0.0000  44.9900 90.3369 528.4589        353.1548        5610.3960       5521.0488
frames/ts_00110_48.0.mrc        6       47.9893         84.7000         71.2800         -5.0000         MotionCorr/job002/frames/ts_00110_48_0_PS.mrc   MotionCorr/job002/frames/ts_00110_48_0_EVN.mrc  MotionCorr/job002/frames/ts_00110_48_0_ODD.mrc  MotionCorr/job002/frames/ts_00110_48_0.mrc      MotionCorr/job002/frames/ts_00110_48_0.star     2.1131  0.0000  2.1131  0.0000  47.9900 90.3369 10.6298 311.2986        5552.9644       5502.8062
frames/ts_00125_51.0.mrc        6       50.9894         84.7000         81.0000         -5.0000         MotionCorr/job002/frames/ts_00125_51_0_PS.mrc   MotionCorr/job002/frames/ts_00125_51_0_EVN.mrc  MotionCorr/job002/frames/ts_00125_51_0_ODD.mrc  MotionCorr/job002/frames/ts_00125_51_0.mrc      MotionCorr/job002/frames/ts_00125_51_0.star     3.0292  0.0000  3.0292  0.0000  50.9900 90.3369 496.9700        651.2730        6818.9727       6709.2480
frames/ts_00130_54.0.mrc        6       53.9901         84.7000         84.2400         -5.0000         MotionCorr/job002/frames/ts_00130_54_0_PS.mrc   MotionCorr/job002/frames/ts_00130_54_0_EVN.mrc  MotionCorr/job002/frames/ts_00130_54_0_ODD.mrc  MotionCorr/job002/frames/ts_00130_54_0.mrc      MotionCorr/job002/frames/ts_00130_54_0.star     1.6235  0.0000  1.6235  0.0000  53.9900 90.3369 20.1884 768.9822        9457.6758       9349.7344
frames/ts_00145_57.0.mrc        6       56.9902         84.7000         93.9600         -5.0000         MotionCorr/job002/frames/ts_00145_57_0_PS.mrc   MotionCorr/job002/frames/ts_00145_57_0_EVN.mrc  MotionCorr/job002/frames/ts_00145_57_0_ODD.mrc  MotionCorr/job002/frames/ts_00145_57_0.mrc      MotionCorr/job002/frames/ts_00145_57_0.star     3.0528  0.0000  3.0528  0.0000  56.9900 90.3369 451.6884        717.7112        46011.7383      45723.8203
frames/ts_00150_60.0.mrc        6       59.9913         84.7000         97.2000         -5.0000         MotionCorr/job002/frames/ts_00150_60_0_PS.mrc   MotionCorr/job002/frames/ts_00150_60_0_EVN.mrc  MotionCorr/job002/frames/ts_00150_60_0_ODD.mrc  MotionCorr/job002/frames/ts_00150_60_0.mrc      MotionCorr/job002/frames/ts_00150_60_0.star     1.4708  0.0000  1.4708  0.0000  59.9900 90.3369 135.0575        570.9975        8106.7886       7868.5649
frames/ts_00165_63.0.mrc        6       62.9905         84.7000         106.9200        -5.0000         MotionCorr/job002/frames/ts_00165_63_0_PS.mrc   MotionCorr/job002/frames/ts_00165_63_0_EVN.mrc  MotionCorr/job002/frames/ts_00165_63_0_ODD.mrc  MotionCorr/job002/frames/ts_00165_63_0.mrc      MotionCorr/job002/frames/ts_00165_63_0.star     3.1333  0.0000  3.1333  0.0000  62.9900 90.3369 431.3717        -162.6895       8025.5352       8014.1846
frames/ts_00170_66.0.mrc        6       65.9911         84.7000         110.1600        -5.0000         MotionCorr/job002/frames/ts_00170_66_0_PS.mrc   MotionCorr/job002/frames/ts_00170_66_0_EVN.mrc  MotionCorr/job002/frames/ts_00170_66_0_ODD.mrc  MotionCorr/job002/frames/ts_00170_66_0.mrc      MotionCorr/job002/frames/ts_00170_66_0.star     1.1700  0.0000  1.1700  0.0000  65.9900 90.3369 233.4375        375.5750        28049.0391      27258.3809

However, using this star file as an input for reconstruction results in the following error message -

in: /path/to/relion/src/jaz/tomography/tomogram_set.cpp, line in:/path/to/relion/src/jaz/tomography/tomogram_set.cpp, line 89
ERROR: 
ERROR: tomogramTable does not contain compulsory rlnMicrographPreExposure labelin: /path/to/relion/src/jaz/tomography/tomogram_set.cpp, line in: /path/to/relion/src/jaz/tomography/tomogram_set.cpp, line 89
ERROR: 
ERROR: tomogramTable does not contain compulsory rlnMicrographPreExposure label

89
ERROR: 
ERROR: tomogramTable does not contain compulsory rlnMicrographPreExposure label
89
ERROR: 
ERROR: tomogramTable does not contain compulsory rlnMicrographPreExposure label

I am running the reconstruct command with the following parameters and the git commit id for this version of relion is 832d07fb.

`which relion_tomo_reconstruct_tomogram_mpi` --t AlignTiltSeries/job004/aligned_tilt_series.star --o ReconstructTomograms/job005/ --tn map-33-A2_ts_001 --generate_split_tomograms  --w 4096 --h 4096 --d 1500 --binned_angpix 7 --noctf  --x0 0 --y0 0 --z0 0  --only_do_unfinished  --j 4  --pipeline_control ReconstructTomograms/job005/

Would really appreciate inputs on where things are going wrong as I am keen to get Relion tomo working in my hands.

EuanPyle commented 11 months ago

Hi Pranav, Those errors tend to come up arbitrarily, they indicate something has gone wrong somewhere but don't really say where. I noticed you're only trying to reconstruct one tomogram as per your job submission, can you empty the Tomogram Name field and see what happens?

EuanPyle commented 11 months ago

Can you also run a CTF Estimation job using the aligned_tilt_series.star as the input and use the output from the CTF estimation job as the input for Reconstruct Tomograms? I can see there's no CTFFIND output info in the tilt series star file which may be messing with things (it adds more columns than just DefocusU/V)

shahpnmlab commented 11 months ago

Hi Euan, I am re-estimating the TS ctf params using CTFfind and will report back. But to add more context to the original post, I was following the pipeline (Import > MotionCorr > CtfFind > Align > Recons). I then went down the rabbit hole of futzing with a test TS star file....

shahpnmlab commented 11 months ago

Ok that worked! So perhaps the AlignTiltSeries job type should be moved to in the third position? The current pipeline implies a workflow that is not congruent with the underlying processing pipeline?

shahpnmlab commented 11 months ago

Ok. So now that I have reconstructed the tomograms in Relion, they appear to be way worse than the ones AreTomo outputs. Am I failing to turn on some kind of flag by any chance?

image

(Left Relion, Right AreTomo)

shahpnmlab commented 11 months ago

Do you think this is because the pre-tilt has not been corrected during TS alignments (So far I have be running everything with default options)? or is there some kind of rotation; X-Y shift convention that is not being respected during imports?

EuanPyle commented 11 months ago

So you should do CTFFind before Align, so the job order is correct. For whatever reason, the output of CTFFIND wasn't contained in your star file used as input to align tilt series most likely.

EuanPyle commented 11 months ago

The other stuff is weird: usually I consistently find that the RELION tomos looks better than the AreTomo ones so I don't quite get that. It's quite hard to judge without direct access to the jobs and stars. Let me think about it a bit 😬

EuanPyle commented 11 months ago

Is that AreTomo rec from the AlignTiltSeries/job[whatever]/external directory?

EuanPyle commented 11 months ago

Also can you post the note.txt from the AlignTiltSeries job plz? And if you've run AreTomo externally to generate that AT rec, what settings you used in AT?