search

asarnow / pyem

Python programs for electron microscopy
GNU General Public License v3.0
114 stars 52 forks source link

star.py --merge-fields does not work to merge rlnImageOriginalName #92

Closed THWeng closed 2 years ago

THWeng commented 2 years ago

Hi,

I have imported signal-subtracted particles from RELION into cryoSPARC for processing, then I wanted to import the particles back to RELION. I have already exported the particles from cryoSPARC using csparc2star.py to star file format (particles_from_cs.star). Now I want to put the unsubtracted particle paths into the file using the command: star.py --merge-key rlnImageName --merge-fields rlnImageOriginalName --merge-source particles_subtracted.star particles_from_cs.star new_particles.star However, the rlnImageOriginalName is always the same as the rlnImageName. I have tried to merge other fields (e.g. rlnAutopickFigureOfMerit) and there was no problem at all. The followings are the star files:

_particles_fromcs.star

data_particles

loop_ _rlnImageName #1 _rlnMicrographName #2 _rlnCoordinateX #3 _rlnCoordinateY #4 _rlnAngleRot #5 _rlnAngleTilt #6 _rlnAnglePsi #7 _rlnOriginXAngst #8 _rlnOriginYAngst #9 _rlnDefocusU #10 _rlnDefocusV #11 _rlnDefocusAngle #12 _rlnPhaseShift #13 _rlnCtfBfactor #14 _rlnOpticsGroup #15 _rlnRandomSubset #16 _rlnClassNumber #17 000001@Subtract/job030/Particles/subtracted_rank1_opticsgroup1.mrcs MotionCorr/job002/Movies/FoilHole_3975900_Data_3974452_3974454_20221004_123909_fractions.mrc 843.958709 2986.206723 37.044888 37.400818 -106.52564 -0.46515 0.736373 17805.140625 16542.193359 145.764862 0.000000 0.000000 1 2 1

_particlessubtracted.star

data_particles

loop_ _rlnCoordinateX #1 _rlnCoordinateY #2 _rlnClassNumber #3 _rlnAutopickFigureOfMerit #4 _rlnAnglePsi #5 _rlnImageName #6 _rlnMicrographName #7 _rlnOpticsGroup #8 _rlnCtfMaxResolution #9 _rlnCtfFigureOfMerit #10 _rlnDefocusU #11 _rlnDefocusV #12 _rlnDefocusAngle #13 _rlnCtfBfactor #14 _rlnCtfScalefactor #15 _rlnPhaseShift #16 _rlnAngleRot #17 _rlnAngleTilt #18 _rlnOriginXAngst #19 _rlnOriginYAngst #20 _rlnNormCorrection #21 _rlnLogLikeliContribution #22 _rlnMaxValueProbDistribution #23 _rlnNrOfSignificantSamples #24 _rlnGroupName #25 _rlnGroupNumber #26 _rlnRandomSubset #27 _rlnImageOriginalName #28 843.958709 2986.206723 1 2.287678 -111.72061 000001@Subtract/job030/Particles/subtracted_rank1_opticsgroup1.mrcs MotionCorr/job002/Movies/FoilHole_3975900_Data_3974452_3974454_20221004_123909_fractions.mrc 1 3.332428 0.143674 17805.140113 16542.193526 145.764870 0.000000 1.000000 0.000000 41.261618 33.558929 -0.05319 0.232865 1.020200 2.106861e+05 0.088977 27 group_21 1 2 1@Polish/job022/Movies/FoilHole_3975900_Data_3974452_3974454_20221004_123909_fractions_shiny.mrcs

_newparticles.star

data_particles

loop_ _rlnImageName #1 _rlnImageOriginalName #2 _rlnMicrographName #3 _rlnCoordinateX #4 _rlnCoordinateY #5 _rlnAngleRot #6 _rlnAngleTilt #7 _rlnAnglePsi #8 _rlnOriginXAngst #9 _rlnOriginYAngst #10 _rlnDefocusU #11 _rlnDefocusV #12 _rlnDefocusAngle #13 _rlnPhaseShift #14 _rlnCtfBfactor #15 _rlnOpticsGroup #16 _rlnRandomSubset #17 _rlnClassNumber #18 000001@Subtract/job030/Particles/subtracted_rank1_opticsgroup1.mrcs 000001@Subtract/job030/Particles/subtracted_rank1_opticsgroup1.mrcs MotionCorr/job002/Movies/FoilHole_3975900_Data_3974452_3974454_20221004_123909_fractions.mrc 843.958709 2986.206723 37.044888 37.400818 -106.525640 -0.465150 0.736373 17805.140625 16542.193359 145.764862 0.000000 0.000000 1 2 1

asarnow commented 2 years ago

Hi @THWeng, I assume your real .star files also have pixel sizes etc. If you use star.py --merge-key ucsfImageIndex,ucsfImagePath --merge-fields ucsfImageOriginalIndex,ucsfImageOriginalPath --merge-source particles_subtracted.star particles_from_cs.star new_particles.star instead, it will work as you intend.

Internally pyem always splits the image index and image path (two parts of rlnImageName) and works with those before recombining them at the end. There are some edge cases like this where a desired change in rlnImage(Original)Name will be ignored. I have to think about the best change to introduce to get the most convenient behavior.

THWeng commented 2 years ago

Hi @asarnow, thank you very much. It works as expected.