EGA-archive / crypt4gh

GA4GH cryptographic tools
http://crypt4gh.readthedocs.io
Apache License 2.0
18 stars 12 forks source link

Batch decrypt? #35

Closed nicoleversetwo closed 1 year ago

nicoleversetwo commented 1 year ago

Is there any way to decrypt multiple files at once? Perhaps I'm missing a simple solution?

Wildcard seems to still only read in one file at at time. I wrote a looping script, but despite setting unlimited timeout, I can't seem to use a script with "expect" either to roll through the files one at a time. We have almost 1000 files to decrypt. Thank you!

silverdaz commented 1 year ago

The CLI is designed to read from stdin and write to stdout. The wildcard is expanded by your shell, and it doesn't play well with what you need to do.

Are you interested in decrypting the 1000 files concurrently?

Python is not a parallel language so even if you write a script that takes the 1000 file names as input (or just a regex, or the wildcard), choose where to output each decrypted file, and loop through the 1000 files using the library (like you suggested), you would then still handle the 1000 files sequentially.

You can adapt your script to interleave the 1000 file decryptions, so you get the illusion of making progress on all the files, but it's still one at a time under the hood.

silverdaz commented 1 year ago

No follow up on this issue. Closing it.