search

warpem / warp

https://warpem.github.io/warp/
GNU General Public License v3.0
41 stars 7 forks source link

WarpTools: Picking and exporting particles from averaged micrographs #221

Open mjaremko opened 3 months ago

mjaremko commented 3 months ago

Is there a better way to pick and export particles from already averaged micrographs?

I was successfully able to picking and generate a star file with this workflow:

WarpTools create_settings --output warp_frameseries.settings --folder_data . --extension ".mrc" --angpix 0.746 mkdir average ln -sr .mrc average WarpTools fs_boxnet_infer --settings warp_frameseries.settings --model ../BoxNet_20220403_191253.pt --diameter 150 --threshold 0.2 --device_list 6 7 WarpTools fs_export_particles --settings warp_frameseries.settings --input matching --output good_particles.star --box 4 --diameter 150 --angpix_coords 0.746 --device_list 6 7

But the 'WarpTools fs_export_particles' command generates the star file with coordinates and 'MicrographName' field, but the ImageName column is empty and now particle stacks are generated. The job does produce a 'Failed to process...' error but there is nothing in the log.

Also, if I process from MotionCorrection>CTFestimation the 'WarpTools fs_export_particles' produces the particle stacks and complete star file.

alisterburt commented 3 months ago

Hi @mjaremko ! Thanks for thoroughly looking at this.

The workflow you're trying to do is not directly supported/expected so it's not surprising that the export fails but it would be nice to make it work - could you share a couple of files with working and not working scripts? This will make fixing the outputs much easier 🙂

mjaremko commented 3 months ago

Hi Alister,

I used a couple .eer movies from EMPIAR-10424. I can't upload here (files are too big), but I can drop them in a link if you have one. Or you can download a couple from EMPIAR. Let me know.

Here are the commands:

EMPIAR 10424 test full

WarpTools create_settings --output warp_frameseries.settings --folder_data Movies --folder_processing warp_frameseries --extension "*.eer" --angpix 0.457 WarpTools fs_motion_and_ctf --settings warp_frameseries.settings c_defocus_min --out_averages --out_average_halves WarpTools fs_boxnet_infer --settings warp_frameseries.settings --model BoxNet_20220403_191253.pt --diameter 150 --threshold 0.2 --device_list 6 7 WarpTools fs_export_particles --settings warp_frameseries.settings --input warp_frameseries/matching/ --output good_particles.star --box 4 --diameter 150 --angpix_coords 0.746 --device_list 6 7

EMPIAR 10424 test from micrographs (no particle stacks)

WarpTools create_settings --output warp_micrographs.settings --folder_data micrographs --folder_processing warp_micrographs --extension ".mrc" --angpix 0.457 mkdir -p warp_micrographs/average ln -sr micrographs/.mrc warp_micrographs/average WarpTools fs_boxnet_infer --settings warp_micrographs.settings --model BoxNet_20220403_191253.pt --diameter 150 --threshold 0.2 --device_list 6 7 WarpTools fs_export_particles --settings warp_micrographs.settings --input warp_micrographs/matching/ --output good_particles.star --box 4 --diameter 150 --angpix_coords 0.746 --device_list 6 7

mjaremko commented 3 months ago

https://www.ebi.ac.uk/empiar/EMPIAR-10424/

alisterburt commented 2 months ago

@mjaremko thanks, could you link to the boxnet weights you are using?

mjaremko commented 2 months ago

Oh right:) You can find it in this zip file for Warp 1.1 beta: https://seafile.unistra.fr/d/44157c1e129544bb9c43/ located here in zipped folder: ./Warp_110_beta1/boxnet3models/BoxNet_20220403_191253.pt

mjaremko commented 2 months ago

Hi Alister, Were you able to access the boxnet weights? Any success with the fixes? Thanks!

alisterburt commented 2 months ago

Hi @mjaremko - apologies but I haven't had time to work on this and fix things for you. I will try to take a look soon but can't promise anything in terms of timeline.

mjaremko commented 2 months ago

Thanks for the update Alister, no worries.