search

ml-struct-bio / cryodrgn

Neural networks for cryo-EM reconstruction
http://cryodrgn.cs.princeton.edu
GNU General Public License v3.0
307 stars 76 forks source link

errors occur when using my own cryosparc .cs .mrc as input #338

Open siayouyang opened 8 months ago

siayouyang commented 8 months ago

data: J489_009_particles.cs (from cryoscparc refine job) *./J487/extract/.mrc** (from cryosparc extract job)

execute: cryodrgn downsample J489_009_particles.cs -D 160 -o J489_particles.160.mrcs --datadir ./J487/extract &

error message： Traceback (most recent call last): File "/home/xyy/anaconda3/envs/cryodrgn/bin/cryodrgn", line 8, in sys.exit(main()) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/main.py", line 74, in main args.func(args) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/commands/downsample.py", line 89, in main old = ImageSource.from_file(args.mrcs, lazy=lazy, indices=ind, datadir=args.datadir) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 75, in from_file return CsSource( File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 429, in init super().init( File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 333, in init self._sources = { File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 334, in filepath: MRCFileSource(filepath) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 238, in init header = MRCHeader.parse(filepath) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/mrc.py", line 112, in parse header = cls(STRUCT.unpack(f.read(1024))) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/mrc.py", line 97, in init self.dtype = DTYPE_FOR_MODE[self.fields["mode"]] KeyError: 1230001760

execute: cryodrgn train_vae J489_009_particles.cs --ctf J489_ctf.pkl --poses J489_poses.pkl --zdim 8 -n 50 --batch-size 8 --uninvert-data -o 01_J489_320 > 01_J489_320.log --datadir ./J487/extract &

error message: (INFO) (train_vae.py) (20-Jan-24 15:17:07) /home/xyy/anaconda3/envs/cryodrgn/bin/cryodrgn train_vae J489_009_particles.cs --ctf J489_ctf.pkl --poses J489_poses.pkl --zdim 8 -n 50 --batch-size 8 --uninvert-data -o 01_J489_320 --datadir ./J487/extract (INFO) (train_vae.py) (20-Jan-24 15:17:07) cryoDRGN 3.1.0b0 (INFO) (train_vae.py) (20-Jan-24 15:17:07) Namespace(particles='/data3/xyy/J489_009_particles.cs', outdir='/data3/xyy/01_J489_320', zdim=8, poses='/data3/xyy/J489_poses.pkl', ctf='/data3/xyy/J489_ctf.pkl', load=None, checkpoint=1, log_interval=1000, verbose=False, seed=4346, ind=None, invert_data=False, window=True, window_r=0.85, datadir='/data3/xyy/J487/extract', lazy=False, shuffler_size=0, num_workers=0, max_threads=16, ntilts=10, random_tilts=False, t_emb_dim=64, tlayers=3, tdim=1024, dose_per_tilt=None, angle_per_tilt=3, num_epochs=50, batch_size=8, wd=0, lr=0.0001, beta=None, beta_control=None, norm=None, amp=True, multigpu=False, do_pose_sgd=False, pretrain=1, emb_type='quat', pose_lr=0.0003, qlayers=3, qdim=1024, encode_mode='resid', enc_mask=None, use_real=False, players=3, pdim=1024, pe_type='gaussian', feat_sigma=0.5, pe_dim=None, domain='fourier', activation='relu', func=<function main at 0x7f42b3b9c550>) (INFO) (train_vae.py) (20-Jan-24 15:17:07) Use cuda True (INFO) (train_vae.py) (20-Jan-24 15:17:07) Loading dataset from /data3/xyy/J489_009_particles.cs Traceback (most recent call last): File "/home/xyy/anaconda3/envs/cryodrgn/bin/cryodrgn", line 8, in sys.exit(main()) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/main.py", line 74, in main args.func(args) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/commands/train_vae.py", line 667, in main data = dataset.ImageDataset( File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/dataset.py", line 36, in init self.src = ImageSource.from_file( File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 75, in from_file return CsSource( File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 429, in init super().init( File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 333, in init self._sources = { File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 334, in filepath: MRCFileSource(filepath) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/source.py", line 238, in init header = MRCHeader.parse(filepath) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/mrc.py", line 112, in parse header = cls(STRUCT.unpack(f.read(1024))) File "/home/xyy/anaconda3/envs/cryodrgn/lib/python3.9/site-packages/cryodrgn/mrc.py", line 97, in init self.dtype = DTYPE_FOR_MODE[self.fields["mode"]] KeyError: 1230001760

siayouyang commented 8 months ago

using preprocess instead of downsample and convert to .star also fail