search

broadinstitute / imaging-backup-scripts

Scripts to backup data for the Imaging Platform
MIT License
1 stars 3 forks source link

Make sure info on file counts, filtering, etc is captured in a file. #22

Open bethac07 opened 2 years ago

bethac07 commented 2 years ago

Originially from here(requires access to the JUMP organization)

I've just added a functionality restore_intelligent will now return the number of non I.T. files in whatever prefix you're restoring, to save[the hassle of tracing why the apparent number of restored files is much less than the number of files in the prefix] in the future. Right now that count though and a lot of other potentially nice information for tracing/sanity checking (how many files did you find in the prefix total, how many were filtered in or out by filter strings) is ONLY printed to the terminal, so now that we potentially see a world where you might be running restore_intelligent on a lot of batches in parallel we may want to think about writing out a summary.txt at the end of each batch that has the information that's currently print only.