Closed frolovsa closed 2 years ago
not needed. md5 sums are available from boto3 interface as
bucket.Object(key).e_tag[1:-1]
or
md5sum = boto3.client('s3').head_object(
Bucket=bucket_name,
Key=resource_name
)['ETag'][1:-1]
compute md5 sum for the file when it is copied. output md5 to a text file at the destination bucket.
This can be done in a separate path on already copied files. can be also combined with the some basic plotting of the data.