search

tbepler / topaz

Pipeline for particle picking in cryo-electron microscopy images using convolutional neural networks trained from positive and unlabeled examples. Also featuring micrograph and tomogram denoising with DNNs.
GNU General Public License v3.0
170 stars 62 forks source link

Support long argument list specified as file for all topaz commands #192

Open nfrasser opened 4 months ago

nfrasser commented 4 months ago

Fixes #47

Use fromfile_prefix_chars option in ArgumentParser to allow specifying a long list of arguments in a file.

Example invocation before (still supported):

topaz denoise -o denoised_micrographs micrographs/mic1.mrc micrographs/mic2.mrc micrographs/mic3.mrc

Example invocation after:

topaz denoise -o denoised_micrographs @mics.txt

Where mics.txt has the following contents:

micrographs/mic1.mrc
micrographs/mic2.mrc
micrographs/mic3.mrc

This would be really beneficial to have when processing thousands of micrographs on systems that have a limit on the maximum number of allowed command-line arguments. It would also be good for debugging purposes e.g., show full command used by CryoSPARC Topaz wrapper instead of a truncated command that excludes the paths for legibility.

@tbepler please let me know if you have any feedback!

tbepler commented 4 months ago

I like this. Was not aware argparse had this option. We need to make sure this doesn't cause any unexpected errors and is compatible with some pending unreleased updates though.

Guillawme commented 4 months ago

This will be a welcome improvement! On several systems, I have run into the "too many command-line arguments" limitation, and there is no other solution than splitting the dataset in several batches to pick separately (pretty annoying to do, and it makes the job graph look more complicated than it really is).