naglepuff
commented
1 month ago As an added option, it might be nice to have a "quietly skip files that aren't in a format we know" flag, since in my large sample files collection (around 60,000+ tiff and dicom files), I have a mix that includes formats that we can't redact (e.g., hdmi, ImageJ, etc). We skip formats we don't recognize quietly (e.g., nd2), and it might be nice in quiet mode to skip any that raise UnsupportedFileTypeError.
Do you have an example of a format, or a file that are disruptive to a run of the program? I thought we were already skipping files that aren't supported.
dgutman
In order to test on a large collection of images, I think it would be nice to be able to do something like
dist/imagedephi plan -r -q /top/level/folderand have a tqdm progress bar. With a single value of the quiet flag (or maybe at the default level), I'd ideally like to see just the files that can't be redacted with the current rule set. The default verbosity is too noisy -- if something failed to redact it has scrolled away. The -q verbosity doesn't show the path of the file that fails.As an added option, it might be nice to have a "quietly skip files that aren't in a format we know" flag, since in my large sample files collection (around 60,000+ tiff and dicom files), I have a mix that includes formats that we can't redact (e.g., hdmi, ImageJ, etc). We skip formats we don't recognize quietly (e.g., nd2), and it might be nice in quiet mode to skip any that raise
UnsupportedFileTypeError.Regarding progress, when we do redactions, we convert the
iter_image_filesgenerator to a list, but on the plan we don't. The list lets tqdm print the percent done, but on a large recursive file set, the check of which files we can possibly read takes a long time. Should we have a progress bar on generating that initial list? That is, a progress bar with unknown total quantity while collection files? Otherwise the program just sits for a very long time.