search

k1LoW / serverless-s3-sync

A plugin to sync local directories and S3 prefixes for Serverless Framework :zap:
182 stars 68 forks source link

Multipart uploads aren't successful #103

Open amang95 opened 2 years ago

amang95 commented 2 years ago

I have seen this issue around Error: ETag does not match MD5 checksum being discussed on some PRs. The above does not seem to be an issue when the object is uploaded as a single request.

But when an object size is large enough to require a multi-part upload, the process errors out with the above error in between uploading parts and hence the file isn't synced to S3 bucket.

For my instance, the file posing a problem is about 17.9 MB in size and fails to upload with the error:

S3 Sync: Syncing directories and S3 prefixes...
. 
 Error ---------------------------------------------------

  Error: ETag does not match MD5 checksum
      at /app/qb-serverless-python/services/smart-builder/node_modules/@auth0/s3/lib/index.js:415:16
      at Pend.wait (/app/qb-serverless-python/services/smart-builder/node_modules/pend/index.js:21:5)
      at Response.<anonymous> (/app/qb-serverless-python/services/smart-builder/node_modules/@auth0/s3/lib/index.js:412:14)
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:367:18)
      at Request.callListeners (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:686:14)
      at Request.transition (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:22:10)
      at AcceptorStateMachine.runTo (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:14:12)
      at /usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:26:10
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:38:9)
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:688:12)
      at Request.callListeners (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:686:14)
      at Request.transition (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:22:10)
      at AcceptorStateMachine.runTo (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:14:12)
      at /usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:26:10
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:38:9)
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:688:12)
      at Request.callListeners (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:686:14)
      at Request.transition (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:22:10)
      at AcceptorStateMachine.runTo (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:14:12)
      at /usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:26:10
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:38:9)
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:688:12)
      at Request.callListeners (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:686:14)
      at Request.transition (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:22:10)
      at AcceptorStateMachine.runTo (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:14:12)
      at /usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:26:10
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:38:9)
      at Request.<anonymous> (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:688:12)
      at Request.callListeners (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
      at callNextListener (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
      at IncomingMessage.onEnd (/usr/lib/node_modules/serverless/node_modules/aws-sdk/lib/event_listeners.js:335:13)
      at IncomingMessage.emit (events.js:412:35)
      at IncomingMessage.emit (domain.js:475:12)
      at endReadableNT (internal/streams/readable.js:1334:12)
      at processTicksAndRejections (internal/process/task_queues.js:82:21)