search

s3tools / s3cmd

Official s3cmd repo -- Command line tool for managing S3 compatible storage services (including Amazon S3 and CloudFront).
https://s3tools.org/s3cmd
GNU General Public License v2.0
4.53k stars 903 forks source link

ERROR: S3 error: 400 (XAmzContentSHA256Mismatch): The provided 'x-amz-content-sha256' header #812

Open saltarob opened 7 years ago

saltarob commented 7 years ago

Hello, I get almost every day following failure message:

ERROR: S3 error: 400 (XAmzContentSHA256Mismatch): The provided 'x-amz-content-sha256' header

What is the reason? How can it be fixed? Thank you.

fviard commented 7 years ago

@AbdulHadiParsdorfer Can you give us a little more context? maybe a debug log. If it is random, are you sure that you don't have network issues and disk or network packet corruptions or something like that??

saltarob commented 7 years ago

Thank you. Here is the debug log:


DEBUG: s3cmd version 1.6.1
DEBUG: ConfigParser: Reading file '/Users/video/.s3cfg'
DEBUG: ConfigParser: access_key->AK...17_chars...A
DEBUG: ConfigParser: access_token->
DEBUG: ConfigParser: add_encoding_exts->
DEBUG: ConfigParser: add_headers->
DEBUG: ConfigParser: bucket_location->EU
DEBUG: ConfigParser: ca_certs_file->
DEBUG: ConfigParser: cache_file->
DEBUG: ConfigParser: check_ssl_certificate->True
DEBUG: ConfigParser: check_ssl_hostname->True
DEBUG: ConfigParser: cloudfront_host->cloudfront.amazonaws.com
DEBUG: ConfigParser: default_mime_type->binary/octet-stream
DEBUG: ConfigParser: delay_updates->False
DEBUG: ConfigParser: delete_after->False
DEBUG: ConfigParser: delete_after_fetch->False
DEBUG: ConfigParser: delete_removed->False
DEBUG: ConfigParser: dry_run->False
DEBUG: ConfigParser: enable_multipart->True
DEBUG: ConfigParser: encoding->UTF-8
DEBUG: ConfigParser: encrypt->False
DEBUG: ConfigParser: expiry_date->
DEBUG: ConfigParser: expiry_days->
DEBUG: ConfigParser: expiry_prefix->
DEBUG: ConfigParser: follow_symlinks->False
DEBUG: ConfigParser: force->False
DEBUG: ConfigParser: get_continue->False
DEBUG: ConfigParser: gpg_command->None
DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
DEBUG: ConfigParser: gpg_passphrase->...-3_chars...
DEBUG: ConfigParser: guess_mime_type->True
DEBUG: ConfigParser: host_base->s3.amazonaws.com
DEBUG: ConfigParser: host_bucket->%(bucket)s.s3.amazonaws.com
DEBUG: ConfigParser: human_readable_sizes->False
DEBUG: ConfigParser: invalidate_default_index_on_cf->False
DEBUG: ConfigParser: invalidate_default_index_root_on_cf->True
DEBUG: ConfigParser: invalidate_on_cf->False
DEBUG: ConfigParser: kms_key->
DEBUG: ConfigParser: limitrate->0
DEBUG: ConfigParser: list_md5->False
DEBUG: ConfigParser: log_target_prefix->
DEBUG: ConfigParser: long_listing->False
DEBUG: ConfigParser: max_delete->-1
DEBUG: ConfigParser: mime_type->
DEBUG: ConfigParser: multipart_chunk_size_mb->15
DEBUG: ConfigParser: multipart_max_chunks->10000
DEBUG: ConfigParser: preserve_attrs->True
DEBUG: ConfigParser: progress_meter->True
DEBUG: ConfigParser: proxy_host->
DEBUG: ConfigParser: proxy_port->0
DEBUG: ConfigParser: put_continue->False
DEBUG: ConfigParser: recursive->False
DEBUG: ConfigParser: recv_chunk->65536
DEBUG: ConfigParser: reduced_redundancy->False
DEBUG: ConfigParser: requester_pays->False
DEBUG: ConfigParser: restore_days->1
DEBUG: ConfigParser: secret_key->gH...37_chars...9
DEBUG: ConfigParser: send_chunk->65536
DEBUG: ConfigParser: server_side_encryption->False
DEBUG: ConfigParser: signature_v2->False
DEBUG: ConfigParser: simpledb_host->sdb.amazonaws.com
DEBUG: ConfigParser: skip_existing->False
DEBUG: ConfigParser: socket_timeout->300
DEBUG: ConfigParser: stats->False
DEBUG: ConfigParser: stop_on_error->False
DEBUG: ConfigParser: storage_class->
DEBUG: ConfigParser: urlencoding_mode->normal
DEBUG: ConfigParser: use_https->True
DEBUG: ConfigParser: use_mime_magic->True
DEBUG: ConfigParser: verbosity->WARNING
DEBUG: ConfigParser: website_endpoint->http://%(bucket)s.s3-website-%(location)s.amazonaws.com/
DEBUG: ConfigParser: website_error->
DEBUG: ConfigParser: website_index->index.html
DEBUG: Updating Config.Config extra_headers[cache-control] -> public,max-age=2052000
DEBUG: Updating Config.Config cache_file -> 
DEBUG: Updating Config.Config follow_symlinks -> False
DEBUG: Updating Config.Config mime_type -> video/mp4
DEBUG: Updating Config.Config verbosity -> 10
DEBUG: Unicodising 'put' using UTF-8
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: Unicodising 's3://saltanat-video-1200/' using UTF-8
DEBUG: Command: put
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
INFO: Compiling list of local files...
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: Unicodising '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: Unicodising '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: Applying --exclude/--include
DEBUG: CHECK: 1980-00-00_en_AdviceForManyQuestions_SN_a.mp4
DEBUG: PASS: u'1980-00-00_en_AdviceForManyQuestions_SN_a.mp4'
INFO: Running stat() and reading/calculating MD5 values on 1 files, this may take some time...
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: doing file I/O to read md5 of 1980-00-00_en_AdviceForManyQuestions_SN_a.mp4
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
INFO: Summary: 1 local files to upload
DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399'}
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: String '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' encoded to '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4'
DEBUG: CreateRequest: resource[uri]=/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?uploads
DEBUG: Using signature v4
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: canonical_headers = cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161226T171319Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
POST
/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4
uploads=
cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161226T171319Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399
x-amz-storage-class:STANDARD

cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399', 'x-amz-date': '20161226T171319Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': 'AWS4-HMAC-SHA256 Credential=XXXXXXXXX/20161226/EU/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=82a96e373ea0caaa80e59e9f5b940ea9e20c37838fd523f241796606547641eb'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: ConnMan.get(): creating new connection: https://saltanat-video-1200.s3.amazonaws.com
DEBUG: Using ca_certs_file None
DEBUG: httplib.HTTPSConnection() has only context
DEBUG: non-proxied HTTPSConnection(saltanat-video-1200.s3.amazonaws.com, None)
DEBUG: format_uri(): /1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?uploads
DEBUG: Sending request method_string='POST', uri='/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?uploads', headers={'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399', 'x-amz-date': '20161226T171319Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': 'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161226/EU/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=82a96e373ea0caaa80e59e9f5b940ea9e20c37838fd523f241796606547641eb'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (https://saltanat-video-1200.s3.amazonaws.com#1)
DEBUG: Response:
{'data': '<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region \'EU\' is wrong; expecting \'eu-west-1\'</Message><Region>eu-west-1</Region><RequestId>F6C712BB5F6A6F21</RequestId><HostId>qOCAr7wmHKeLw6oevpOOaZ2A58maTZJdao4/x74RYt5cB7q94MWa0iqHMlW96KJejEWu9ES6OPY=</HostId></Error>',
'headers': {'connection': 'close',
            'content-type': 'application/xml',
            'date': 'Mon, 26 Dec 2016 17:13:20 GMT',
            'server': 'AmazonS3',
            'transfer-encoding': 'chunked',
            'x-amz-id-2': 'qOCAr7wmHKeLw6oevpOOaZ2A58maTZJdao4/x74RYt5cB7q94MWa0iqHMlW96KJejEWu9ES6OPY=',
            'x-amz-request-id': 'F6C712BB5F6A6F21'},
'reason': 'Bad Request',
'status': 400}
INFO: Forwarding request to eu-west-1
DEBUG: Using signature v4
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: canonical_headers = cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161226T171320Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
POST
/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4
uploads=
cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161226T171320Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399
x-amz-storage-class:STANDARD

cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399', 'x-amz-date': '20161226T171320Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': u'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161226/eu-west-1/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=a78dba9c108bc77160d6e7ad5ff5f4aebabe25349ac9283f9a8dd49b374d7c0b'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: ConnMan.get(): re-using connection: https://saltanat-video-1200.s3.amazonaws.com#1
DEBUG: format_uri(): /1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?uploads
DEBUG: Sending request method_string='POST', uri='/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?uploads', headers={'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482772399/atime:1482772386/md5:71ea1a9ca54a57e5e7cb00f9d049f54c/ctime:1482772399', 'x-amz-date': '20161226T171320Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': u'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161226/eu-west-1/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=a78dba9c108bc77160d6e7ad5ff5f4aebabe25349ac9283f9a8dd49b374d7c0b'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (https://saltanat-video-1200.s3.amazonaws.com#2)
DEBUG: Response:
{'data': '<?xml version="1.0" encoding="UTF-8"?>\n<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>saltanat-video-1200</Bucket><Key>1980-00-00_en_AdviceForManyQuestions_SN_a.mp4</Key><UploadId>JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH</UploadId></InitiateMultipartUploadResult>',
'headers': {'date': 'Mon, 26 Dec 2016 17:13:22 GMT',
            'server': 'AmazonS3',
            'transfer-encoding': 'chunked',
            'x-amz-id-2': 'IwJ85Tyk1i4MzuCFPiBJ8quiM5fmcFD3zuteuTaGEVK83UKARUjNdMUhkUZ2zaySwpmwAcB7Rkg=',
            'x-amz-request-id': '146C5DF36F228735'},
'reason': 'OK',
'status': 200}
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: MultiPart: Uploading /Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4 in 4 parts
DEBUG: Uploading part 1 of u'JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH' (15728640 bytes)
DEBUG: String '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' encoded to '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4'
DEBUG: CreateRequest: resource[uri]=/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?partNumber=1&uploadId=JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
upload: '/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' -> 's3://saltanat-video-1200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4'  [part 1 of 4, 15MB] [1 of 1]
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: canonical_headers = content-length:15728640
host:saltanat-video-1200.s3.amazonaws.com
x-amz-content-sha256:84a49595a920a0867095877f4c951e26fdbfcbd0940d8f81a5abdc2833ae09f5
x-amz-date:20161226T171322Z

DEBUG: Canonical Request:
PUT
/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4
partNumber=1&uploadId=JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH
content-length:15728640
host:saltanat-video-1200.s3.amazonaws.com
x-amz-content-sha256:84a49595a920a0867095877f4c951e26fdbfcbd0940d8f81a5abdc2833ae09f5
x-amz-date:20161226T171322Z

content-length;host;x-amz-content-sha256;x-amz-date
84a49595a920a0867095877f4c951e26fdbfcbd0940d8f81a5abdc2833ae09f5
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': '84a49595a920a0867095877f4c951e26fdbfcbd0940d8f81a5abdc2833ae09f5', 'content-length': '15728640', 'Authorization': u'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161226/eu-west-1/s3/aws4_request,SignedHeaders=content-length;host;x-amz-content-sha256;x-amz-date,Signature=4261c35b5d3aa105c2c62a6556259813c6ccd8d7d9141fed79267642ba20b3f3', 'x-amz-date': '20161226T171322Z'}
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: ConnMan.get(): re-using connection: https://saltanat-video-1200.s3.amazonaws.com#2
DEBUG: format_uri(): /1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?partNumber=1&uploadId=JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH

   65536 of 15728640     0% in    1s    56.50 kB/s
  262144 of 15728640     1% in    3s    81.69 kB/s
  458752 of 15728640     2% in    4s    93.89 kB/s
  589824 of 15728640     3% in    6s    94.91 kB/s
  786432 of 15728640     5% in    7s   100.77 kB/s
  917504 of 15728640     5% in    8s   101.47 kB/s
 1048576 of 15728640     6% in   10s   101.91 kB/s
 1179648 of 15728640     7% in   11s   102.33 kB/s
 1310720 of 15728640     8% in   12s   101.85 kB/s
 1441792 of 15728640     9% in   13s   101.92 kB/s
 1572864 of 15728640    10% in   15s   102.10 kB/s
 1703936 of 15728640    10% in   16s   102.49 kB/s
 1835008 of 15728640    11% in   17s   102.63 kB/s
 1966080 of 15728640    12% in   18s   102.79 kB/s
 2097152 of 15728640    13% in   19s   102.91 kB/s
 2228224 of 15728640    14% in   21s   103.01 kB/s
 2359296 of 15728640    15% in   22s   103.17 kB/s
 2490368 of 15728640    15% in   23s   102.66 kB/s
 2621440 of 15728640    16% in   24s   103.45 kB/s
 2752512
 of 15728640    17% in   25s   103.49 kB/s
 2883584 of 15728640    18% in   27s   103.66 kB/s
 3014656 of 15728640    19% in   28s   103.75 kB/s
 3145728 of 15728640    20% in   29s   103.82 kB/s
 3276800 of 15728640    20% in   30s   103.61 kB/s
 3407872 of 15728640    21% in   32s   102.74 kB/s
 3604480 of 15728640    22% in   33s   103.74 kB/s
 3735552 of 15728640    23% in   35s   103.79 kB/s
 3866624 of 15728640    24% in   36s   103.86 kB/s
 3997696 of 15728640    25% in   37s   103.91 kB/s
 4128768 of 15728640    26% in   38s   103.96 kB/s
 4259840 of 15728640    27% in   39s   104.01 kB/s
 4390912 of 15728640    27% in   41s   103.63 kB/s
 4521984 of 15728640    28% in   42s   103.67 kB/s
 4653056 of 15728640    29% in   44s   101.28 kB/s
 4849664 of 15728640    30% in   46s   102.87 kB/s
 4980736 of 15728640    31% in   47s   102.93 kB/s
 5111808 of 15728640    32% in   48s   103.06 kB/s
 5242880 of 15728640    33% in   49s   103.12 kB/s
 5373952 of 15728
640    34% in   50s   103.14 kB/s
 5505024 of 15728640    35% in   52s   103.19 kB/s
 5636096 of 15728640    35% in   53s   103.03 kB/s
 5767168 of 15728640    36% in   54s   103.30 kB/s
 5898240 of 15728640    37% in   55s   103.35 kB/s
 5963776 of 15728640    37% in   56s   102.62 kB/s
 6160384 of 15728640    39% in   58s   102.81 kB/s
 6356992 of 15728640    40% in   59s   103.52 kB/s
 6488064 of 15728640    41% in   61s   103.56 kB/s
 6619136 of 15728640    42% in   62s   103.21 kB/s
 6815744 of 15728640    43% in   64s   102.95 kB/s
 7012352 of 15728640    44% in   66s   103.71 kB/s
 7143424 of 15728640    45% in   67s   103.06 kB/s
 7340032 of 15728640    46% in   69s   103.79 kB/s
 7471104 of 15728640    47% in   70s   103.26 kB/s
 7667712 of 15728640    48% in   72s   103.85 kB/s
 7798784 of 15728640    49% in   73s   103.88 kB/s
 7929856 of 15728640    50% in   74s   103.91 kB/s
 8060928 of 15728640    51% in   75s   103.75 kB/s
 8192000 of 15728640    52
% in   77s   103.65 kB/s
 8323072 of 15728640    52% in   78s   103.57 kB/s
 8454144 of 15728640    53% in   79s   103.72 kB/s
 8585216 of 15728640    54% in   80s   103.77 kB/s
 8716288 of 15728640    55% in   82s   103.53 kB/s
 8781824 of 15728640    55% in   83s   102.90 kB/s
 8912896 of 15728640    56% in   84s   102.87 kB/s
 9043968 of 15728640    57% in   85s   102.90 kB/s
 9175040 of 15728640    58% in   87s   102.94 kB/s
 9306112 of 15728640    59% in   88s   102.98 kB/s
 9437184 of 15728640    60% in   89s   103.02 kB/s
 9568256 of 15728640    60% in   90s   102.95 kB/s
 9699328 of 15728640    61% in   92s   102.01 kB/s
 9830400 of 15728640    62% in   94s   101.79 kB/s
 9961472 of 15728640    63% in   95s   101.61 kB/s
10092544 of 15728640    64% in   97s   101.47 kB/s
10158080 of 15728640    64% in   98s   100.62 kB/s
10289152 of 15728640    65% in  100s   100.09 kB/s
10420224 of 15728640    66% in  101s    99.98 kB/s
10551296 of 15728640    67% in  103s    99.96 kB/s
10682368 of 15728640    67% in  104s    99.99 kB/s
10813440 
of 15728640    68% in  105s    99.95 kB/s
10878976 of 15728640    69% in  106s    99.36 kB/s
11010048 of 15728640    70% in  108s    98.81 kB/s
11141120 of 15728640    70% in  110s    98.76 kB/s
11272192 of 15728640    71% in  111s    98.76 kB/s
11337728 of 15728640    72% in  112s    98.31 kB/s
11403264 of 15728640    72% in  113s    97.85 kB/s
11468800 of 15728640    72% in  115s    97.37 kB/s
11599872 of 15728640    73% in  116s    97.22 kB/s
11730944 of 15728640    74% in  117s    97.30 kB/s
11862016 of 15728640    75% in  118s    97.38 kB/s
11993088 of 15728640    76% in  120s    97.43 kB/s
12124160 of 15728640    77% in  121s    97.22 kB/s
12255232 of 15728640    77% in  123s    97.23 kB/s
12386304 of 15728640    78% in  124s    97.34 kB/s
12517376 of 15728640    79% in  125s    97.37 kB/s
12648448 of 15728640    80% in  126s    97.43 kB/s
12779520 of 15728640    81% in  128s    97.50 kB/s
12910592 of 15728640    82% in  129s    97.55 kB/s
13041664 of 157286
40    82% in  130s    97.55 kB/s
13172736 of 15728640    83% in  132s    97.26 kB/s
13303808 of 15728640    84% in  133s    97.46 kB/s
13434880 of 15728640    85% in  134s    97.46 kB/s
13565952 of 15728640    86% in  135s    97.49 kB/s
13697024 of 15728640    87% in  137s    97.51 kB/s
13828096 of 15728640    87% in  138s    97.47 kB/s
13959168 of 15728640    88% in  139s    97.49 kB/s
14090240 of 15728640    89% in  141s    97.41 kB/s
14221312 of 15728640    90% in  142s    97.44 kB/s
14352384 of 15728640    91% in  143s    97.45 kB/s
14483456 of 15728640    92% in  145s    97.38 kB/s
14614528 of 15728640    92% in  146s    97.42 kB/s
14745600 of 15728640    93% in  147s    97.44 kB/s
14876672 of 15728640    94% in  149s    97.12 kB/s
15007744 of 15728640    95% in  151s    97.00 kB/s
15138816 of 15728640    96% in  152s    97.02 kB/s
15269888 of 15728640    97% in  154s    96.76 kB/s
15400960 of 15728640    97% in  155s    96.56 kB/s
15532032 of 15728640    98%
 in  156s    96.65 kB/s
15663104 of 15728640    99% in  158s    96.65 kB/sDEBUG: ConnMan.put(): connection put back to pool (https://saltanat-video-1200.s3.amazonaws.com#3)
DEBUG: Response: {'status': 200, 'headers': {'content-length': '0', 'x-amz-id-2': 'jCiDfmDr/hsmkuj8BeIaVIaWkk2C3VxO3aa2IsVoIKR6lk6KPQXtk0e4wEuin+DrLRtMYrzNTW8=', 'server': 'AmazonS3', 'x-amz-request-id': '9AC9509FE4855F0A', 'etag': '"d3f32381c3bd5e504f516a4893ea8320"', 'date': 'Mon, 26 Dec 2016 17:13:37 GMT'}, 'reason': 'OK', 'data': '', 'size': 15728640L}

15728640 of 15728640   100% in  160s    95.80 kB/s
15728640 of 15728640   100% in  160s    95.80 kB/s  done
DEBUG: MD5 sums: computed=d3f32381c3bd5e504f516a4893ea8320, received="d3f32381c3bd5e504f516a4893ea8320"
DEBUG: Uploading part 2 of u'JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH' (15728640 bytes)
DEBUG: String '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' encoded to '1980-00-00_en_AdviceForManyQuestions_SN_a.mp4'
DEBUG: CreateRequest: resource[uri]=/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?partNumber=2&uploadId=JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
upload: '/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' -> 's3://saltanat-video-1200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4'  [part 2 of 4, 15MB] [1 of 1]
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: canonical_headers = content-length:15728640
host:saltanat-video-1200.s3.amazonaws.com
x-amz-content-sha256:5f57f049608e45983231fc7c14bf17acecb1f65f9f93f6add0227d5368f099de
x-amz-date:20161226T171602Z

DEBUG: Canonical Request:
PUT
/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4
partNumber=2&uploadId=JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH
content-length:15728640
host:saltanat-video-1200.s3.amazonaws.com
x-amz-content-sha256:5f57f049608e45983231fc7c14bf17acecb1f65f9f93f6add0227d5368f099de
x-amz-date:20161226T171602Z

content-length;host;x-amz-content-sha256;x-amz-date
5f57f049608e45983231fc7c14bf17acecb1f65f9f93f6add0227d5368f099de
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': '5f57f049608e45983231fc7c14bf17acecb1f65f9f93f6add0227d5368f099de', 'content-length': '15728640', 'Authorization': u'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161226/eu-west-1/s3/aws4_request,SignedHeaders=content-length;host;x-amz-content-sha256;x-amz-date,Signature=88f52292fded01204273c02d614307a2b211ea6e39dccc667d4402a82d7c345d', 'x-amz-date': '20161226T171602Z'}
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: ConnMan.get(): re-using connection: https://saltanat-video-1200.s3.amazonaws.com#3
DEBUG: format_uri(): /1980-00-00_en_AdviceForManyQuestions_SN_a.mp4?partNumber=2&uploadId=JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH

   65536 of 15728640     0% in    0s   260.06 kB/s
  196608 of 15728640     1% in    1s   147.10 kB/s
  262144 of 15728640     1% in    2s    94.50 kB/s
  393216 of 15728640     2% in    4s    88.88 kB/s
  524288 of 15728640     3% in    5s    90.92 kB/s
  655360 of 15728640     4% in    6s    92.69 kB/s
  786432 of 15728640     5% in    8s    91.20 kB/s
  917504 of 15728640     5% in    9s    91.18 kB/s
 1048576 of 15728640     6% in   11s    92.66 kB/s
 1179648 of 15728640     7% in   12s    93.63 kB/s
 1310720 of 15728640     8% in   13s    93.34 kB/s
 1441792 of 15728640     9% in   15s    92.98 kB/s
 1572864 of 15728640    10% in   16s    93.52 kB/s
 1703936 of 15728640    10% in   17s    93.90 kB/s
 1835008 of 15728640    11% in   18s    94.51 kB/s
 1966080 of 15728640    12% in   20s    94.98 kB/s
 2097152 of 15728640    13% in   21s    95.37 kB/s
 2228224 of 15728640    14% in   22s    95.52 kB/s
 2359296 of 15728640    15% in   24s    95.49 kB/s
 2490368
 of 15728640    15% in   25s    93.78 kB/s
 2621440 of 15728640    16% in   27s    93.81 kB/s
 2752512 of 15728640    17% in   28s    94.44 kB/s
 2883584 of 15728640    18% in   29s    94.85 kB/s
 3014656 of 15728640    19% in   30s    95.15 kB/s
 3145728 of 15728640    20% in   32s    95.03 kB/s
 3276800 of 15728640    20% in   33s    95.20 kB/s
 3407872 of 15728640    21% in   34s    95.45 kB/s
 3538944 of 15728640    22% in   36s    95.73 kB/s
 3670016 of 15728640    23% in   37s    95.10 kB/s
 3801088 of 15728640    24% in   39s    94.33 kB/s
 3932160 of 15728640    25% in   40s    93.83 kB/s
 4063232 of 15728640    25% in   42s    94.02 kB/s
 4194304 of 15728640    26% in   43s    94.19 kB/s
 4325376 of 15728640    27% in   44s    94.41 kB/s
 4456448 of 15728640    28% in   46s    94.03 kB/s
 4653056 of 15728640    29% in   48s    93.42 kB/s
 4849664 of 15728640    30% in   49s    94.78 kB/s
 4980736 of 15728640    31% in   51s    95.01 kB/s
 5111808 of 15728
640    32% in   52s    95.24 kB/s
 5242880 of 15728640    33% in   53s    95.46 kB/s
 5373952 of 15728640    34% in   54s    95.65 kB/s
 5505024 of 15728640    35% in   56s    95.20 kB/s
 5701632 of 15728640    36% in   57s    96.02 kB/s
 5832704 of 15728640    37% in   59s    96.20 kB/s
 5963776 of 15728640    37% in   60s    96.39 kB/s
 6094848 of 15728640    38% in   61s    96.57 kB/s
 6225920 of 15728640    39% in   63s    96.42 kB/s
 6422528 of 15728640    40% in   64s    96.99 kB/s
 6553600 of 15728640    41% in   65s    97.16 kB/s
 6684672 of 15728640    42% in   67s    97.31 kB/s
 6815744 of 15728640    43% in   68s    97.48 kB/s
 6946816 of 15728640    44% in   69s    97.16 kB/s
 7143424 of 15728640    45% in   71s    97.83 kB/s
 7274496 of 15728640    46% in   72s    97.93 kB/s
 7405568 of 15728640    47% in   73s    98.05 kB/s
 7536640 of 15728640    47% in   74s    98.18 kB/s
 7667712 of 15728640    48% in   76s    98.31 kB/s
 7798784 of 15728640    49
% in   77s    98.42 kB/s
 7929856 of 15728640    50% in   78s    98.48 kB/s
 8060928 of 15728640    51% in   79s    98.63 kB/s
 8192000 of 15728640    52% in   81s    98.29 kB/s
 8388608 of 15728640    53% in   82s    98.91 kB/s
 8519680 of 15728640    54% in   84s    98.95 kB/s
 8650752 of 15728640    55% in   85s    99.09 kB/s
 8781824 of 15728640    55% in   86s    99.16 kB/s
 8912896 of 15728640    56% in   87s    99.25 kB/s
 9043968 of 15728640    57% in   88s    99.34 kB/s
 9175040 of 15728640    58% in   90s    99.32 kB/s
 9306112 of 15728640    59% in   91s    99.33 kB/s
 9437184 of 15728640    60% in   92s    99.10 kB/s
 9568256 of 15728640    60% in   94s    99.23 kB/s
 9699328 of 15728640    61% in   95s    99.31 kB/s
 9830400 of 15728640    62% in   96s    99.39 kB/s
 9961472 of 15728640    63% in   97s    99.38 kB/s
10092544 of 15728640    64% in   99s    99.52 kB/s
10223616 of 15728640    65% in  100s    99.44 kB/s
10354688 of 15728640    65% in  101s    99.67 kB/s
10485760 of 15728640    66% in  102s    99.74 kB/s
10616832 
of 15728640    67% in  103s    99.81 kB/s
10747904 of 15728640    68% in  105s    99.73 kB/s
10878976 of 15728640    69% in  106s    99.80 kB/s
11010048 of 15728640    70% in  107s    99.86 kB/s
11141120 of 15728640    70% in  108s    99.92 kB/s
11206656 of 15728640    71% in  109s    99.59 kB/s
11403264 of 15728640    72% in  111s   100.02 kB/s
11534336 of 15728640    73% in  112s   100.10 kB/s
11665408 of 15728640    74% in  113s   100.16 kB/s
11796480 of 15728640    75% in  115s   100.15 kB/s
11927552 of 15728640    75% in  116s   100.19 kB/s
12058624 of 15728640    76% in  117s   100.04 kB/s
12189696 of 15728640    77% in  118s   100.06 kB/s
12320768 of 15728640    78% in  120s   100.11 kB/s
12451840 of 15728640    79% in  121s   100.14 kB/s
12582912 of 15728640    80% in  122s   100.16 kB/s
12713984 of 15728640    80% in  123s   100.21 kB/s
12845056 of 15728640    81% in  125s   100.24 kB/s
12976128 of 15728640    82% in  126s   100.25 kB/s
13107200 of 157286
40    83% in  127s   100.29 kB/s
13238272 of 15728640    84% in  128s   100.33 kB/s
13369344 of 15728640    85% in  130s   100.33 kB/s
13500416 of 15728640    85% in  131s   100.30 kB/s
13631488 of 15728640    86% in  132s   100.30 kB/s
13762560 of 15728640    87% in  133s   100.32 kB/s
13893632 of 15728640    88% in  135s   100.32 kB/s
14024704 of 15728640    89% in  136s   100.43 kB/s
14155776 of 15728640    90% in  137s   100.44 kB/s
14286848 of 15728640    90% in  138s   100.50 kB/s
14417920 of 15728640    91% in  140s   100.53 kB/s
14548992 of 15728640    92% in  141s   100.57 kB/s
14680064 of 15728640    93% in  142s   100.61 kB/s
14811136 of 15728640    94% in  143s   100.65 kB/s
14942208 of 15728640    95% in  145s   100.56 kB/s
15073280 of 15728640    95% in  146s   100.62 kB/s
15204352 of 15728640    96% in  147s   100.47 kB/s
15400960 of 15728640    97% in  149s   100.71 kB/s
15532032 of 15728640    98% in  150s   100.76 kB/s
15663104 of 15728640    99%
 in  151s   100.80 kB/sDEBUG: ConnMan.put(): connection put back to pool (https://saltanat-video-1200.s3.amazonaws.com#4)
DEBUG: Response: {'status': 400, 'headers': {'x-amz-id-2': 't+zO88+UniJFb0uoxQIwLdNcwz/1SVLSpMmule8vVK0XUa0HkK+6ElprpIi/lcP2v96HMyL35IE=', 'server': 'AmazonS3', 'transfer-encoding': 'chunked', 'connection': 'close', 'x-amz-request-id': '55C47858FB58B343', 'date': 'Mon, 26 Dec 2016 17:18:34 GMT', 'content-type': 'application/xml'}, 'reason': 'Bad Request', 'data': '<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>XAmzContentSHA256Mismatch</Code><Message>The provided \'x-amz-content-sha256\' header does not match what was computed.</Message><ClientComputedContentSHA256>5f57f049608e45983231fc7c14bf17acecb1f65f9f93f6add0227d5368f099de</ClientComputedContentSHA256><S3ComputedContentSHA256>29652d3a3dfbe81d0f73f5b95f10cfd292c3cc983137ce78aee861a2c53414c5</S3ComputedContentSHA256><RequestId>55C47858FB58B343</RequestId><HostId>t+zO88+UniJFb0uoxQIwLdNcwz/1SVLSpMmule8vVK0XUa0HkK+6ElprpIi/lcP2v96HMyL35IE=</HostId></Error>', 'size': 15728640L}

15728640 of 15728640   100% in  153s   100.03 kB/s
15728640 of 15728640   100% in  153s   100.03 kB/s  done
DEBUG: S3Error: 400 (Bad Request)
DEBUG: HttpHeader: x-amz-id-2: t+zO88+UniJFb0uoxQIwLdNcwz/1SVLSpMmule8vVK0XUa0HkK+6ElprpIi/lcP2v96HMyL35IE=
DEBUG: HttpHeader: server: AmazonS3
DEBUG: HttpHeader: transfer-encoding: chunked
DEBUG: HttpHeader: connection: close
DEBUG: HttpHeader: x-amz-request-id: 55C47858FB58B343
DEBUG: HttpHeader: date: Mon, 26 Dec 2016 17:18:34 GMT
DEBUG: HttpHeader: content-type: application/xml
DEBUG: ErrorXML: Code: 'XAmzContentSHA256Mismatch'
DEBUG: ErrorXML: Message: "The provided 'x-amz-content-sha256' header does not match what was computed."
DEBUG: ErrorXML: ClientComputedContentSHA256: '5f57f049608e45983231fc7c14bf17acecb1f65f9f93f6add0227d5368f099de'
DEBUG: ErrorXML: S3ComputedContentSHA256: '29652d3a3dfbe81d0f73f5b95f10cfd292c3cc983137ce78aee861a2c53414c5'
DEBUG: ErrorXML: RequestId: '55C47858FB58B343'
DEBUG: ErrorXML: HostId: 't+zO88+UniJFb0uoxQIwLdNcwz/1SVLSpMmule8vVK0XUa0HkK+6ElprpIi/lcP2v96HMyL35IE='
ERROR: 
Upload of '/Volumes/Assets/02 Publish/01 mp4/2200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4' part 2 failed. Use
 /usr/local/bin/s3cmd abortmp s3://saltanat-video-1200/1980-00-00_en_AdviceForManyQuestions_SN_a.mp4 JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH
to abort the upload, or
 /usr/local/bin/s3cmd --upload-id JLXI_vRMdFVsQ3scBPcsT7G95Bhx0HL2dArD1_DGHd3jU7PG0fMjFdmct7vSulGy1EJZJoxFUxuJsK55Zr2slmFRfyHDeY39svc1uR95ygrBqr9.Cf4W0iuIe57VEgFH put ...
to continue the upload.
ERROR: S3 error: 400 (XAmzContentSHA256Mismatch): The provided 'x-amz-content-sha256' header does not match what was computed.
fviard commented 7 years ago

Thanks. It could be interesting if you can run the command another time so we can see if it always fail at the same point, otherwise the new log for the transfer of the same file can show us if it is the s3cmd calculus that is wrong or transmission issues.

saltarob commented 7 years ago

Thank you. I always make three different qualities of the same video. The other two videos don't get this problem. Only the biggest one always gets the Mismatch failure. Here is the second log.

DEBUG: s3cmd version 1.6.1
DEBUG: ConfigParser: Reading file '/Users/video/.s3cfg'
DEBUG: ConfigParser: access_key->AK...17_chars...A
DEBUG: ConfigParser: access_token->
DEBUG: ConfigParser: add_encoding_exts->
DEBUG: ConfigParser: add_headers->
DEBUG: ConfigParser: bucket_location->EU
DEBUG: ConfigParser: ca_certs_file->
DEBUG: ConfigParser: cache_file->
DEBUG: ConfigParser: check_ssl_certificate->True
DEBUG: ConfigParser: check_ssl_hostname->True
DEBUG: ConfigParser: cloudfront_host->cloudfront.amazonaws.com
DEBUG: ConfigParser: default_mime_type->binary/octet-stream
DEBUG: ConfigParser: delay_updates->False
DEBUG: ConfigParser: delete_after->False
DEBUG: ConfigParser: delete_after_fetch->False
DEBUG: ConfigParser: delete_removed->False
DEBUG: ConfigParser: dry_run->False
DEBUG: ConfigParser: enable_multipart->True
DEBUG: ConfigParser: encoding->UTF-8
DEBUG: ConfigParser: encrypt->False
DEBUG: ConfigParser: expiry_date->
DEBUG: ConfigParser: expiry_days->
DEBUG: ConfigParser: expiry_prefix->
DEBUG: ConfigParser: follow_symlinks->False
DEBUG: ConfigParser: force->False
DEBUG: ConfigParser: get_continue->False
DEBUG: ConfigParser: gpg_command->None
DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
DEBUG: ConfigParser: gpg_passphrase->...-3_chars...
DEBUG: ConfigParser: guess_mime_type->True
DEBUG: ConfigParser: host_base->s3.amazonaws.com
DEBUG: ConfigParser: host_bucket->%(bucket)s.s3.amazonaws.com
DEBUG: ConfigParser: human_readable_sizes->False
DEBUG: ConfigParser: invalidate_default_index_on_cf->False
DEBUG: ConfigParser: invalidate_default_index_root_on_cf->True
DEBUG: ConfigParser: invalidate_on_cf->False
DEBUG: ConfigParser: kms_key->
DEBUG: ConfigParser: limitrate->0
DEBUG: ConfigParser: list_md5->False
DEBUG: ConfigParser: log_target_prefix->
DEBUG: ConfigParser: long_listing->False
DEBUG: ConfigParser: max_delete->-1
DEBUG: ConfigParser: mime_type->
DEBUG: ConfigParser: multipart_chunk_size_mb->15
DEBUG: ConfigParser: multipart_max_chunks->10000
DEBUG: ConfigParser: preserve_attrs->True
DEBUG: ConfigParser: progress_meter->True
DEBUG: ConfigParser: proxy_host->
DEBUG: ConfigParser: proxy_port->0
DEBUG: ConfigParser: put_continue->False
DEBUG: ConfigParser: recursive->False
DEBUG: ConfigParser: recv_chunk->65536
DEBUG: ConfigParser: reduced_redundancy->False
DEBUG: ConfigParser: requester_pays->False
DEBUG: ConfigParser: restore_days->1
DEBUG: ConfigParser: secret_key->gH...37_chars...9
DEBUG: ConfigParser: send_chunk->65536
DEBUG: ConfigParser: server_side_encryption->False
DEBUG: ConfigParser: signature_v2->False
DEBUG: ConfigParser: simpledb_host->sdb.amazonaws.com
DEBUG: ConfigParser: skip_existing->False
DEBUG: ConfigParser: socket_timeout->300
DEBUG: ConfigParser: stats->False
DEBUG: ConfigParser: stop_on_error->False
DEBUG: ConfigParser: storage_class->
DEBUG: ConfigParser: urlencoding_mode->normal
DEBUG: ConfigParser: use_https->True
DEBUG: ConfigParser: use_mime_magic->True
DEBUG: ConfigParser: verbosity->WARNING
DEBUG: ConfigParser: website_endpoint->http://%(bucket)s.s3-website-%(location)s.amazonaws.com/
DEBUG: ConfigParser: website_error->
DEBUG: ConfigParser: website_index->index.html
DEBUG: Updating Config.Config extra_headers[cache-control] -> public,max-age=2052000
DEBUG: Updating Config.Config cache_file -> 
DEBUG: Updating Config.Config follow_symlinks -> False
DEBUG: Updating Config.Config mime_type -> video/mp4
DEBUG: Updating Config.Config verbosity -> 10
DEBUG: Unicodising 'put' using UTF-8
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: Unicodising 's3://saltanat-video-1200/' using UTF-8
DEBUG: Command: put
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
INFO: Compiling list of local files...
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: Unicodising '1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: Unicodising '1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: Applying --exclude/--include
DEBUG: CHECK: 1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4
DEBUG: PASS: u'1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4'
INFO: Running stat() and reading/calculating MD5 values on 1 files, this may take some time...
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: doing file I/O to read md5 of 1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
INFO: Summary: 1 local files to upload
DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271'}
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: String '1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' encoded to '1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4'
DEBUG: CreateRequest: resource[uri]=/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4?uploads
DEBUG: Using signature v4
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: canonical_headers = cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161228T164113Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
POST
/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4
uploads=
cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161228T164113Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271
x-amz-storage-class:STANDARD

cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271', 'x-amz-date': '20161228T164113Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': 'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161228/EU/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=197632ce2721980213cc732ec7b2e1cd3b7d3f163c91a39cf6e82219331ca179'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: ConnMan.get(): creating new connection: https://saltanat-video-1200.s3.amazonaws.com
DEBUG: Using ca_certs_file None
DEBUG: httplib.HTTPSConnection() has only context
DEBUG: non-proxied HTTPSConnection(saltanat-video-1200.s3.amazonaws.com, None)
DEBUG: format_uri(): /1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4?uploads
DEBUG: Sending request method_string='POST', uri='/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4?uploads', headers={'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271', 'x-amz-date': '20161228T164113Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': 'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161228/EU/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=197632ce2721980213cc732ec7b2e1cd3b7d3f163c91a39cf6e82219331ca179'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (https://saltanat-video-1200.s3.amazonaws.com#1)
DEBUG: Response:
{'data': '<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region \'EU\' is wrong; expecting \'eu-west-1\'</Message><Region>eu-west-1</Region><RequestId>EF304CCE5ED35087</RequestId><HostId>7S6oMp1dQtyrkto1UpoEb0zlc9O7cdSEiUevu0TRx3+lMD4KNgcz2L8rn6JLUPJUL4jOaZcq1Lk=</HostId></Error>',
'headers': {'connection': 'close',
            'content-type': 'application/xml',
            'date': 'Wed, 28 Dec 2016 16:41:15 GMT',
            'server': 'AmazonS3',
            'transfer-encoding': 'chunked',
            'x-amz-id-2': '7S6oMp1dQtyrkto1UpoEb0zlc9O7cdSEiUevu0TRx3+lMD4KNgcz2L8rn6JLUPJUL4jOaZcq1Lk=',
            'x-amz-request-id': 'EF304CCE5ED35087'},
'reason': 'Bad Request',
'status': 400}
INFO: Forwarding request to eu-west-1
DEBUG: Using signature v4
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: canonical_headers = cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161228T164116Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
POST
/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4
uploads=
cache-control:public,max-age=2052000
content-type:video/mp4
host:saltanat-video-1200.s3.amazonaws.com
x-amz-acl:public-read
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20161228T164116Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271
x-amz-storage-class:STANDARD

cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271', 'x-amz-date': '20161228T164116Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': u'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161228/eu-west-1/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=f9f337fa69f781cf5cb766d277bc116536d54caae517f8d3657b1d4d241b8bd7'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: ConnMan.get(): re-using connection: https://saltanat-video-1200.s3.amazonaws.com#1
DEBUG: format_uri(): /1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4?uploads
DEBUG: Sending request method_string='POST', uri='/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4?uploads', headers={'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-acl': 'public-read', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:video/gid:20/mode:33206/mtime:1482943271/atime:1482943272/md5:752c3b256e315aae0a2ca608c6737059/ctime:1482943271', 'x-amz-date': '20161228T164116Z', 'cache-control': 'public,max-age=2052000', 'content-type': 'video/mp4', 'Authorization': u'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161228/eu-west-1/s3/aws4_request,SignedHeaders=cache-control;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=f9f337fa69f781cf5cb766d277bc116536d54caae517f8d3657b1d4d241b8bd7'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (https://saltanat-video-1200.s3.amazonaws.com#2)
DEBUG: Response:
{'data': '<?xml version="1.0" encoding="UTF-8"?>\n<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>saltanat-video-1200</Bucket><Key>1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4</Key><UploadId>G5JVV9qcy59v_3wvsKycy9AIsF9IklY2J.yfTjoKK8VSvlB2wjOJqMxt5cw6mqlGRHzBG9ZuKGLvYIaeY1JquzKSut9y5VvOuQ3Tyx6qwjBoiZ2IQDsOEq4DsIEZk9b5</UploadId></InitiateMultipartUploadResult>',
'headers': {'date': 'Wed, 28 Dec 2016 16:41:19 GMT',
            'server': 'AmazonS3',
            'transfer-encoding': 'chunked',
            'x-amz-id-2': 'eDyIvAfXcj4ncg/+4fRDp9Roi6MQHgfFU/uraciljFL41EMOdzYFXyB9BIhnIiwxJdsVskrZ5m4=',
            'x-amz-request-id': 'D3D59475368E8668'},
'reason': 'OK',
'status': 200}
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: MultiPart: Uploading /Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4 in 20 parts
DEBUG: Uploading part 1 of u'G5JVV9qcy59v_3wvsKycy9AIsF9IklY2J.yfTjoKK8VSvlB2wjOJqMxt5cw6mqlGRHzBG9ZuKGLvYIaeY1JquzKSut9y5VvOuQ3Tyx6qwjBoiZ2IQDsOEq4DsIEZk9b5' (15728640 bytes)
DEBUG: String '1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' encoded to '1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4'
DEBUG: CreateRequest: resource[uri]=/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4?partNumber=1&uploadId=G5JVV9qcy59v_3wvsKycy9AIsF9IklY2J.yfTjoKK8VSvlB2wjOJqMxt5cw6mqlGRHzBG9ZuKGLvYIaeY1JquzKSut9y5VvOuQ3Tyx6qwjBoiZ2IQDsOEq4DsIEZk9b5
DEBUG: Unicodising '/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
upload: '/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' -> 's3://saltanat-video-1200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4'  [part 1 of 20, 15MB] [1 of 1]
DEBUG: DeUnicodising u'/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: canonical_headers = content-length:15728640
host:saltanat-video-1200.s3.amazonaws.com
x-amz-content-sha256:2f82d72c35ed7f96b38a17f6d78463508de8ac7954c040a69191946db24cdb65
x-amz-date:20161228T164118Z

DEBUG: Canonical Request:
PUT
/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4
partNumber=1&uploadId=G5JVV9qcy59v_3wvsKycy9AIsF9IklY2J.yfTjoKK8VSvlB2wjOJqMxt5cw6mqlGRHzBG9ZuKGLvYIaeY1JquzKSut9y5VvOuQ3Tyx6qwjBoiZ2IQDsOEq4DsIEZk9b5
content-length:15728640
host:saltanat-video-1200.s3.amazonaws.com
x-amz-content-sha256:2f82d72c35ed7f96b38a17f6d78463508de8ac7954c040a69191946db24cdb65
x-amz-date:20161228T164118Z

content-length;host;x-amz-content-sha256;x-amz-date
2f82d72c35ed7f96b38a17f6d78463508de8ac7954c040a69191946db24cdb65
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': '2f82d72c35ed7f96b38a17f6d78463508de8ac7954c040a69191946db24cdb65', 'content-length': '15728640', 'Authorization': u'AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20161228/eu-west-1/s3/aws4_request,SignedHeaders=content-length;host;x-amz-content-sha256;x-amz-date,Signature=38363657b8981d1e21384eb5077852238e2c095c4dddea71ff6212938c18b044', 'x-amz-date': '20161228T164118Z'}
DEBUG: get_hostname(saltanat-video-1200): saltanat-video-1200.s3.amazonaws.com
DEBUG: ConnMan.get(): re-using connection: https://saltanat-video-1200.s3.amazonaws.com#2
DEBUG: format_uri(): /1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4?partNumber=1&uploadId=G5JVV9qcy59v_3wvsKycy9AIsF9IklY2J.yfTjoKK8VSvlB2wjOJqMxt5cw6mqlGRHzBG9ZuKGLvYIaeY1JquzKSut9y5VvOuQ3Tyx6qwjBoiZ2IQDsOEq4DsIEZk9b5

   65536 of 15728640     0% in    0s   214.57 kB/s
  196608 of 15728640     1% in    3s    60.67 kB/s
  262144 of 15728640     1% in    4s    59.85 kB/s
  393216 of 15728640     2% in    5s    69.13 kB/s
  524288 of 15728640     3% in    7s    72.23 kB/s
  720896 of 15728640     4% in    9s    77.14 kB/s
  851968 of 15728640     5% in   10s    81.40 kB/s
  983040 of 15728640     6% in   11s    81.16 kB/s
 1114112 of 15728640     7% in   13s    82.71 kB/s
 1245184 of 15728640     7% in   14s    84.49 kB/s
 1376256 of 15728640     8% in   15s    86.06 kB/s
 1507328 of 15728640     9% in   16s    87.51 kB/s
 1638400 of 15728640    10% in   18s    84.96 kB/s
 1835008 of 15728640    11% in   20s    88.73 kB/s
 1966080 of 15728640    12% in   21s    89.68 kB/s
 2097152 of 15728640    13% in   22s    90.52 kB/s
 2228224 of 15728640    14% in   23s    91.27 kB/s
 2359296 of 15728640    15% in   25s    91.97 kB/s
 2490368 of 15728640    15% in   26s    91.93 kB/s
 2621440
 of 15728640    16% in   27s    93.12 kB/s
 2752512 of 15728640    17% in   28s    93.65 kB/s
 2883584 of 15728640    18% in   29s    94.13 kB/s
 3014656 of 15728640    19% in   31s    92.92 kB/s
 3145728 of 15728640    20% in   32s    93.23 kB/s
 3276800 of 15728640    20% in   34s    93.74 kB/s
 3407872 of 15728640    21% in   35s    93.99 kB/s
 3538944 of 15728640    22% in   36s    94.34 kB/s
 3670016 of 15728640    23% in   38s    93.63 kB/s
 3866624 of 15728640    24% in   39s    94.98 kB/s
 3997696 of 15728640    25% in   40s    95.28 kB/s
 4128768 of 15728640    26% in   42s    95.56 kB/s
 4259840 of 15728640    27% in   43s    95.76 kB/s
 4390912 of 15728640    27% in   45s    95.29 kB/s
 4587520 of 15728640    29% in   46s    96.42 kB/s
 4718592 of 15728640    30% in   48s    95.97 kB/s
 4849664 of 15728640    30% in   49s    96.09 kB/s
 4980736 of 15728640    31% in   50s    95.80 kB/s
 5111808 of 15728640    32% in   51s    96.22 kB/s
 5242880 of 15728
640    33% in   53s    96.43 kB/s
 5373952 of 15728640    34% in   54s    96.65 kB/s
 5505024 of 15728640    35% in   55s    96.72 kB/s
 5636096 of 15728640    35% in   56s    96.89 kB/s
 5767168 of 15728640    36% in   58s    96.19 kB/s
 5898240 of 15728640    37% in   59s    96.44 kB/s
 6029312 of 15728640    38% in   60s    96.63 kB/s
 6160384 of 15728640    39% in   62s    96.81 kB/s
 6291456 of 15728640    40% in   63s    96.96 kB/s
 6422528 of 15728640    40% in   64s    97.11 kB/s
 6488064 of 15728640    41% in   65s    96.52 kB/s
 6619136 of 15728640    42% in   67s    96.47 kB/s
 6750208 of 15728640    42% in   68s    96.49 kB/s
 6881280 of 15728640    43% in   70s    95.93 kB/s
 7077888 of 15728640    45% in   71s    96.63 kB/s
 7208960 of 15728640    45% in   72s    96.76 kB/s
 7340032 of 15728640    46% in   74s    96.47 kB/s
 7536640 of 15728640    47% in   75s    97.07 kB/s
 7667712 of 15728640    48% in   77s    96.98 kB/s
 7798784 of 15728640    49
% in   78s    96.84 kB/s
 7929856 of 15728640    50% in   79s    96.98 kB/s
 8060928 of 15728640    51% in   81s    96.68 kB/s
 8257536 of 15728640    52% in   82s    97.21 kB/s
 8388608 of 15728640    53% in   84s    97.22 kB/s
 8519680 of 15728640    54% in   85s    97.28 kB/s
 8585216 of 15728640    54% in   86s    96.84 kB/s
 8781824 of 15728640    55% in   87s    97.47 kB/s
 8912896 of 15728640    56% in   89s    97.42 kB/s
 9043968 of 15728640    57% in   90s    97.62 kB/s
 9175040 of 15728640    58% in   91s    97.70 kB/s
 9240576 of 15728640    58% in   92s    97.33 kB/s
 9371648 of 15728640    59% in   94s    97.26 kB/s
 9502720 of 15728640    60% in   95s    97.36 kB/s
 9633792 of 15728640    61% in   96s    97.46 kB/s
 9764864 of 15728640    62% in   97s    97.55 kB/s
 9895936 of 15728640    62% in   99s    97.25 kB/s
10092544 of 15728640    64% in  100s    97.59 kB/s
10223616 of 15728640    65% in  102s    97.66 kB/s
10354688 of 15728640    65% in  103s    97.75 kB/s
10485760 of 15728640    66% in  104s    97.83 kB/s
10616832 
of 15728640    67% in  105s    97.93 kB/s
10747904 of 15728640    68% in  107s    97.98 kB/s
10878976 of 15728640    69% in  108s    98.04 kB/s
11010048 of 15728640    70% in  109s    97.96 kB/s
11141120 of 15728640    70% in  110s    98.19 kB/s
11272192 of 15728640    71% in  112s    98.26 kB/s
11403264 of 15728640    72% in  113s    98.32 kB/s
11534336 of 15728640    73% in  114s    98.39 kB/s
11599872 of 15728640    73% in  115s    98.00 kB/s
11796480 of 15728640    75% in  116s    98.48 kB/s
11927552 of 15728640    75% in  118s    98.54 kB/s
12058624 of 15728640    76% in  119s    98.61 kB/s
12189696 of 15728640    77% in  120s    98.66 kB/s
12320768 of 15728640    78% in  122s    98.47 kB/s
12451840 of 15728640    79% in  123s    98.55 kB/s
12582912 of 15728640    80% in  124s    98.61 kB/s
12713984 of 15728640    80% in  125s    98.64 kB/s
12845056 of 15728640    81% in  127s    98.74 kB/s
12976128 of 15728640    82% in  128s    98.65 kB/s
13107200 of 157286
40    83% in  129s    98.69 kB/s
13238272 of 15728640    84% in  130s    98.71 kB/s
13369344 of 15728640    85% in  132s    98.76 kB/s
13500416 of 15728640    85% in  133s    98.80 kB/s
13631488 of 15728640    86% in  134s    98.81 kB/s
13762560 of 15728640    87% in  135s    98.83 kB/s
13893632 of 15728640    88% in  137s    98.73 kB/s
14024704 of 15728640    89% in  138s    98.90 kB/s
14155776 of 15728640    90% in  139s    98.84 kB/s
14286848 of 15728640    90% in  141s    98.81 kB/s
14417920 of 15728640    91% in  142s    98.86 kB/s
14548992 of 15728640    92% in  143s    98.87 kB/s
14614528 of 15728640    92% in  144s    98.53 kB/s
14811136 of 15728640    94% in  146s    98.79 kB/s
14942208 of 15728640    95% in  147s    98.83 kB/s
15073280 of 15728640    95% in  149s    98.50 kB/s
15204352 of 15728640    96% in  150s    98.63 kB/s
15335424 of 15728640    97% in  152s    98.26 kB/s
15400960 of 15728640    97% in  153s    97.95 kB/s
15466496 of 15728640    98%
 in  154s    97.61 kB/s
15597568 of 15728640    99% in  156s    97.45 kB/s
15728640 of 15728640   100% in  157s    97.51 kB/sDEBUG: ConnMan.put(): connection put back to pool (https://saltanat-video-1200.s3.amazonaws.com#3)
DEBUG: Response: {'status': 400, 'headers': {'x-amz-id-2': 'W1pt++EUlE/wDl/vYRzGO0vybm3TRYhhRnkIr6Jj1uB6PyhQQezH96ZvGMxIB7iiyA+LtFFclxA=', 'server': 'AmazonS3', 'transfer-encoding': 'chunked', 'connection': 'close', 'x-amz-request-id': 'F1C7BC441BAA4DE3', 'date': 'Wed, 28 Dec 2016 16:43:56 GMT', 'content-type': 'application/xml'}, 'reason': 'Bad Request', 'data': '<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>XAmzContentSHA256Mismatch</Code><Message>The provided \'x-amz-content-sha256\' header does not match what was computed.</Message><ClientComputedContentSHA256>2f82d72c35ed7f96b38a17f6d78463508de8ac7954c040a69191946db24cdb65</ClientComputedContentSHA256><S3ComputedContentSHA256>b0747b3d190628165d7b8d92202565b6bfc48d12768bc0cc531410e784d4d59c</S3ComputedContentSHA256><RequestId>F1C7BC441BAA4DE3</RequestId><HostId>W1pt++EUlE/wDl/vYRzGO0vybm3TRYhhRnkIr6Jj1uB6PyhQQezH96ZvGMxIB7iiyA+LtFFclxA=</HostId></Error>', 'size': 15728640L}

15728640 of 15728640   100% in  158s    96.74 kB/s
15728640 of 15728640   100% in  158s    96.74 kB/s  done
DEBUG: S3Error: 400 (Bad Request)
DEBUG: HttpHeader: x-amz-id-2: W1pt++EUlE/wDl/vYRzGO0vybm3TRYhhRnkIr6Jj1uB6PyhQQezH96ZvGMxIB7iiyA+LtFFclxA=
DEBUG: HttpHeader: server: AmazonS3
DEBUG: HttpHeader: transfer-encoding: chunked
DEBUG: HttpHeader: connection: close
DEBUG: HttpHeader: x-amz-request-id: F1C7BC441BAA4DE3
DEBUG: HttpHeader: date: Wed, 28 Dec 2016 16:43:56 GMT
DEBUG: HttpHeader: content-type: application/xml
DEBUG: ErrorXML: Code: 'XAmzContentSHA256Mismatch'
DEBUG: ErrorXML: Message: "The provided 'x-amz-content-sha256' header does not match what was computed."
DEBUG: ErrorXML: ClientComputedContentSHA256: '2f82d72c35ed7f96b38a17f6d78463508de8ac7954c040a69191946db24cdb65'
DEBUG: ErrorXML: S3ComputedContentSHA256: 'b0747b3d190628165d7b8d92202565b6bfc48d12768bc0cc531410e784d4d59c'
DEBUG: ErrorXML: RequestId: 'F1C7BC441BAA4DE3'
DEBUG: ErrorXML: HostId: 'W1pt++EUlE/wDl/vYRzGO0vybm3TRYhhRnkIr6Jj1uB6PyhQQezH96ZvGMxIB7iiyA+LtFFclxA='
ERROR: 
Upload of '/Volumes/Assets/02 Publish/01 mp4/2200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4' part 1 failed. Use
 /usr/local/bin/s3cmd abortmp s3://saltanat-video-1200/1989-12-22_tr_SerefiniziKoruyun_SN_a.mp4 G5JVV9qcy59v_3wvsKycy9AIsF9IklY2J.yfTjoKK8VSvlB2wjOJqMxt5cw6mqlGRHzBG9ZuKGLvYIaeY1JquzKSut9y5VvOuQ3Tyx6qwjBoiZ2IQDsOEq4DsIEZk9b5
to abort the upload, or
 /usr/local/bin/s3cmd --upload-id G5JVV9qcy59v_3wvsKycy9AIsF9IklY2J.yfTjoKK8VSvlB2wjOJqMxt5cw6mqlGRHzBG9ZuKGLvYIaeY1JquzKSut9y5VvOuQ3Tyx6qwjBoiZ2IQDsOEq4DsIEZk9b5 put ...
to continue the upload.
ERROR: S3 error: 400 (XAmzContentSHA256Mismatch): The provided 'x-amz-content-sha256' header does not match what was computed.
fviard commented 6 years ago

@AbdulHadiParsdorfer A lot of things have been fixed in s3cmd, are you still encountering these issues? and if yes, have you tried with the last version 2.0.1 to see if it is still there?

fstab50 commented 6 years ago

I am getting the exact same error with v2.0.1 today when attempting to upload a 55 GB vmdk image that I'm backing up to s3. Fails on Part 414 out of 557 (100 MB chunk size, encryption with KMS key, below).

System:

Appreciate any suggestions or remedies you may have.

upload: '/home/blake/VirtualMachines/Windows7-x86_64/Windows7-x86_64.vmdk' -> 's3://glacier-virtualmachines/2017-12-21_Windows7-x86_64/Windows7-x86_64.vmdk'  [part 412 of 557, 100MB] [8 of 9]
 104857600 of 104857600   100% in   14s     7.06 MB/s  done
upload: '/home/blake/VirtualMachines/Windows7-x86_64/Windows7-x86_64.vmdk' -> 's3://glacier-virtualmachines/2017-12-21_Windows7-x86_64/Windows7-x86_64.vmdk'  [part 413 of 557, 100MB] [8 of 9]
 104857600 of 104857600   100% in   14s     7.04 MB/s  done
upload: '/home/blake/VirtualMachines/Windows7-x86_64/Windows7-x86_64.vmdk' -> 's3://glacier-virtualmachines/2017-12-21_Windows7-x86_64/Windows7-x86_64.vmdk'  [part 414 of 557, 100MB] [8 of 9]
 104857600 of 104857600   100% in   14s     7.13 MB/s  done
ERROR: 
Upload of '/home/blake/VirtualMachines/Windows7-x86_64/Windows7-x86_64.vmdk' part 414 failed. Use
  /usr/local/bin/s3cmd abortmp s3://glacier-virtualmachines/2017-12-21_Windows7-x86_64/Windows7-x86_64.vmdk B9HyXP1onRrDvDjZHlmYng7BcuzJXzSmqBKGBNbbCIi2762OqfRjj.rLMwFjkRJdasLBEKW3WeuQmZl5hKBzwxOxa0Y5ct_e_zW.n7fo7lyDIVaJn7gmTK5NMl1Pd8Kv
to abort the upload, or
  /usr/local/bin/s3cmd --upload-id B9HyXP1onRrDvDjZHlmYng7BcuzJXzSmqBKGBNbbCIi2762OqfRjj.rLMwFjkRJdasLBEKW3WeuQmZl5hKBzwxOxa0Y5ct_e_zW.n7fo7lyDIVaJn7gmTK5NMl1Pd8Kv put ...
to continue the upload.
ERROR: S3 error: 400 (XAmzContentSHA256Mismatch): The provided 'x-amz-content-sha256' header does not match what was computed.
saltarob commented 6 years ago

I updated to the latest version and still get it and also getting much more timeout errors now with the same error code 400.