feathersjs-ecosystem / feathers-blob

Feathers service for blob storage, like S3.
http://feathersjs.com
MIT License
92 stars 32 forks source link

The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. #97

Closed palmtown closed 1 year ago

palmtown commented 1 year ago

Steps to reproduce

(First please check that this issue is not already solved as described here)

After upgrading AWS SDK to "@aws-sdk/client-s3": "3.224.0", feathers-blob stopped downloading files and now gives the error "[ERR_INVALID_ARG_TYPE]: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of IncomingMessage." Is this related to https://github.com/feathersjs-ecosystem/feathers-blob/issues/96 ?

Expected behavior

feathers-blob should return the downloaded file.

Actual behavior

An error was generated, "[ERR_INVALID_ARG_TYPE]: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of IncomingMessage"

Here is the full trace:

TypeError [ERR_INVALID_ARG_TYPE]: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of IncomingMessage at new NodeError (node:internal/errors:387:5) at readableAddChunk (node:internal/streams/readable:260:13) at S3Readable.Readable.push (node:internal/streams/readable:228:10) at SimpleQueue.processed [as _callback] (/home/dev/node_modules/s3-download-stream/index.js:47:10) at /home/dev/node_modules/SimpleQueue/SimpleQueue.js:31:9 at /home/dev/node_modules/s3-download-stream/index.js:67:5 at /home/dev/node_modules/@aws-sdk/smithy-client/dist-cjs/client.js:16:35 at processTicksAndRejections (node:internal/process/task_queues:96:5) { code: 'ERR_INVALID_ARG_TYPE' }

System configuration

Tell us about the applicable parts of your setup. I'm using feathers-blob integrated with s3

Module versions (especially the part that's not working): feathers-blob 2.6.0

NodeJS version:

Operating System:

Browser Version:

React Native Version:

Module Loader:

claustres commented 1 year ago

Hi @palmtown, AWS SDK v3 is a big change if you'd like to use the new API based on commands. As you noticed it causes some problems in feathers-blob from our experience and would require underlying dependencies to be partly rewritten (typically https://github.com/jb55/s3-blob-store). Indeed, the AWS SDK is not a direct dependency of feathers blob, which relies on the abstract blob store interface.

If you want to really use AWS SDK v3 I suggest you have a look to https://github.com/kalisio/feathers-s3 instead. Otherwise you should also be able to use AWS SDK v2 for feathers blob and AWS SDK v3 in other part of your code, although not really clean.

palmtown commented 1 year ago

Hello @claustres

First, I appreciate you taking the time to provide insight and recommendations, much appreciated. Secondly, I have decided to run both versions of AWS to continue to support in the interim and wait to see what the future holds for feathers-blob. I am a fan of the work and it has helped me move my development in the right direction.

Again, thanks and enjoy your day.