Compare directories md5s

broadinstitute / tgg_methods

Repo for miscellaneous methods developed by the methods group that don't fit anywhere else

MIT License

4 stars 0 forks source link

Compare directories md5s #15

Closed mike-w-wilson closed 3 years ago

mike-w-wilson commented 4 years ago

This script compares copied files md5s. The inputs are a single file, containing the origin directory and destination directory, the top level google bucket where the input file lives, and an output file within that bucket. The directories being compared cannot be renamed as the paths are needed for dictionary lookups. I have tested this on a 3GB directory that was copied from gnomad-public to gnomad-public-requester-pays and it took 35 minutes to run.

mike-w-wilson commented 3 years ago

Closing this PR as #28 is more efficient.