search

iterate-ch / cyberduck

Cyberduck is a libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift file transfer client for Mac and Windows.
https://cyberduck.io/
GNU General Public License v3.0
3.33k stars 290 forks source link

Cause of S3 EntityTooSmall bad request #16430

Open AliveDevil opened 3 weeks ago

AliveDevil commented 3 weeks ago

Creation of UploadId:

INFO  ch.cyberduck.transcript.request - PUT /~%24Name.pptx?uploadId=2~QJnUL26aj5d-WWsEENMv3_uiNUG5EnC&partNumber=2 HTTP/1.1
INFO  ch.cyberduck.transcript.request - Content-MD5: +u4XTs5Em8pTrqMSnZJQaQ==
INFO  ch.cyberduck.transcript.request - Content-Type: application/octet-stream
INFO  ch.cyberduck.transcript.request - x-amz-content-sha256: f23391587f1c9fc48eabd1e95f4caf16f585ef09941b7bc24f023d228e81ccd5
INFO  ch.cyberduck.transcript.request - Content-Length: 110
INFO  ch.cyberduck.transcript.request - Host: test.s3-testing.lab.xc
INFO  ch.cyberduck.transcript.request - Connection: Keep-Alive
INFO  ch.cyberduck.transcript.request - User-Agent: Mountain Duck/4.16.4.22409 (Windows 10/10.0.22631.0) (amd64)
INFO  ch.cyberduck.transcript.request - Accept-Encoding: gzip,deflate
INFO  ch.cyberduck.transcript.request - Expect: 100-continue
INFO  ch.cyberduck.transcript.request - x-amz-date: 20241016T112746Z
INFO  ch.cyberduck.transcript.request - Authorization: ********
INFO  ch.cyberduck.transcript.response - HTTP/1.1 200 OK
INFO  ch.cyberduck.transcript.response - Content-Length: 0
INFO  ch.cyberduck.transcript.response - ETag: "faee174ece449bca53aea3129d925069"
INFO  ch.cyberduck.transcript.response - Accept-Ranges: bytes
INFO  ch.cyberduck.transcript.response - x-amz-request-id: tx00000d50bff54b923b897-00670fa332-1a943d48-default
INFO  ch.cyberduck.transcript.response - Date: Wed, 16 Oct 2024 11:27:46 GMT
INFO  ch.cyberduck.transcript.response - Connection: Keep-Alive
DEBUG org.apache.http.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection [id: 10][route: {}->http://test.s3-testing.lab.xc:80] can be kept alive indefinitely
DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-10: set socket timeout to 0
DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 10][route: {}->http://test.s3-testing.lab.xc:80][total available: 2; route allocated: 5 of 2147483647; total allocated: 6 of 2147483647]
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Response for 'PUT'. Content-Type: , Headers: [Content-Length: 0, ETag: "faee174ece449bca53aea3129d925069", Accept-Ranges: bytes, x-amz-request-id: tx00000d50bff54b923b897-00670fa332-1a943d48-default, Date: Wed, 16 Oct 2024 11:27:46 GMT, Connection: Keep-Alive]
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Response entity: [Content-Length: 0,Chunked: false]
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Entity length: 0
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Received response code 200; matches one of expected set ([200, 201, 204])? true
DEBUG org.jets3t.service.utils.ServiceUtils - Processing REST metadata items
DEBUG org.jets3t.service.utils.ServiceUtils - Removed header prefix x-amz- from key: x-amz-content-sha256=>content-sha256
DEBUG org.jets3t.service.utils.ServiceUtils - Ignoring metadata item: Accept-Ranges=bytes
DEBUG org.jets3t.service.utils.ServiceUtils - Leaving header item unchanged: ETag="faee174ece449bca53aea3129d925069"
DEBUG org.jets3t.service.utils.ServiceUtils - Ignoring metadata item: Connection=Keep-Alive
DEBUG org.jets3t.service.utils.ServiceUtils - Removed header prefix x-amz- from key: x-amz-request-id=>request-id
DEBUG org.jets3t.service.utils.ServiceUtils - Leaving HTTP header item unchanged: Content-Length=110
DEBUG org.jets3t.service.utils.ServiceUtils - Parsing date string 'Wed, 16 Oct 2024 11:27:46 GMT' into Date object for key: Date
DEBUG org.jets3t.service.utils.ServiceUtils - Leaving header item unchanged: Date=Wed Oct 16 13:27:46 CEST 2024
DEBUG org.jets3t.service.utils.ServiceUtils - Leaving HTTP header item unchanged: Content-MD5=+u4XTs5Em8pTrqMSnZJQaQ==
DEBUG org.jets3t.service.utils.ServiceUtils - Leaving HTTP header item unchanged: Content-Type=application/octet-stream
DEBUG ch.cyberduck.core.s3.S3MultipartWriteFeature - Saved object Path{path='/test/~$Name.pptx', type=[file]} with checksum faee174ece449bca53aea3129d925069
DEBUG ch.cyberduck.core.io.StreamCopier - End of file reached with 110 bytes read from stream

Upload part:

INFO  ch.cyberduck.transcript.request - POST /~%24Name.pptx?uploadId=2~QJnUL26aj5d-WWsEENMv3_uiNUG5EnC HTTP/1.1
INFO  ch.cyberduck.transcript.request - Content-Type: text/plain
INFO  ch.cyberduck.transcript.request - Content-Length: 321
INFO  ch.cyberduck.transcript.request - Host: test.s3-testing.lab.xc
INFO  ch.cyberduck.transcript.request - Connection: Keep-Alive
INFO  ch.cyberduck.transcript.request - User-Agent: Mountain Duck/4.16.4.22409 (Windows 10/10.0.22631.0) (amd64)
INFO  ch.cyberduck.transcript.request - Accept-Encoding: gzip,deflate
INFO  ch.cyberduck.transcript.request - x-amz-date: 20241016T112748Z
INFO  ch.cyberduck.transcript.request - x-amz-content-sha256: 5fb3f488745859c402f58097b6cc909444ec5a6516043a7526dc655315782fea
INFO  ch.cyberduck.transcript.request - Authorization: ********
INFO  ch.cyberduck.transcript.response - HTTP/1.1 400 Bad Request
INFO  ch.cyberduck.transcript.response - Content-Length: 243
INFO  ch.cyberduck.transcript.response - x-amz-request-id: tx000007ba163e552c717e5-00670fa334-1a943d48-default
INFO  ch.cyberduck.transcript.response - Accept-Ranges: bytes
INFO  ch.cyberduck.transcript.response - Content-Type: application/xml
INFO  ch.cyberduck.transcript.response - Date: Wed, 16 Oct 2024 11:27:48 GMT
INFO  ch.cyberduck.transcript.response - Connection: Keep-Alive
DEBUG org.apache.http.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Response for 'POST'. Content-Type: application/xml, Headers: [Content-Length: 243, x-amz-request-id: tx000007ba163e552c717e5-00670fa334-1a943d48-default, Accept-Ranges: bytes, Content-Type: application/xml, Date: Wed, 16 Oct 2024 11:27:48 GMT, Connection: Keep-Alive]
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Response entity: ResponseEntityProxy{[Content-Type: application/xml,Content-Length: 243,Chunked: false]}
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Entity length: 243
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Received response code 400; matches one of expected set ([200])? false
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Error response xml: true
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Error response entity: ResponseEntityProxy{[Content-Type: application/xml,Content-Length: 243,Chunked: false]}
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Error response entity length: 243
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Response '/~%24Name.pptx' - Received error response with XML message
DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection [id: 10][route: {}->http://test.s3-testing.lab.xc:80] can be kept alive indefinitely
DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-10: set socket timeout to 0
DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 10][route: {}->http://test.s3-testing.lab.xc:80][total available: 2; route allocated: 5 of 2147483647; total allocated: 6 of 2147483647]
DEBUG org.jets3t.service.impl.rest.httpclient.HttpMethodReleaseInputStream - Released HttpMethod as its response data stream is fully consumed
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Rethrowing as a ServiceException error in performRequest: org.jets3t.service.ServiceException: Service Error Message. -- ResponseCode: 400, ResponseStatus: null, XML Error Message: <?xml version="1.0" encoding="UTF-8"?><Error><Code>EntityTooSmall</Code><Message></Message><BucketName>test</BucketName><RequestId>tx000007ba163e552c717e5-00670fa334-1a943d48-default</RequestId><HostId>1a943d48-default-default</HostId></Error>
DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Releasing HttpClient connection after error: Service Error Message.
dkocher commented 3 weeks ago

This will not be reproducible with Cyberduck where only uploads larger than 100MB ^1 will use the multipart upload API.