search

ZJONSSON / node-unzipper

node.js cross-platform unzip using streams
Other
435 stars 116 forks source link

Build fails without @aws-sdk/client-s3 package in package.json #330

Open MaksimDedov opened 1 month ago

MaksimDedov commented 1 month ago

Hey! When I install the unzipper npm package and don't install separately @aws-sdk/client-s3 I get the build exception like below. Please move @aws-sdk/client-s3 to required dependencies for automatic installation, or fix it usage.

Thanks in advance

ERROR in ./node_modules/unzipper/lib/Open/index.js 97:52-81
Module not found: Error: Can't resolve '@aws-sdk/client-s3' in 'E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\Open'
resolve '@aws-sdk/client-s3' in 'E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\Open'
  Parsed request is a module
  using description file: E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\package.json (relative path: ./lib/Open)
    resolve as module
      E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\Open\node_modules doesn't exist or is not a directory
      E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\node_modules doesn't exist or is not a directory
      looking for modules in E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules
        single file module
          using description file: E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\package.json (relative path: ./node_modules/@aws-sdk/client-s3)
            no extension
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3 doesn't exist
            .ts
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.ts doesn't exist
            .js
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.js doesn't exist
            .json
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.json doesn't exist
            .node
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.node doesn't exist
        E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3 doesn't exist
      E:\Work\finsemble-mono\packages\electron-adapter\node_modules\node_modules doesn't exist or is not a directory
      looking for modules in E:\Work\finsemble-mono\packages\electron-adapter\node_modules  
        single file module
          using description file: E:\Work\finsemble-mono\packages\electron-adapter\package.json (relative path: ./node_modules/@aws-sdk/client-s3)
            no extension
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3 doesn't exist
            .ts
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.ts doesn't exist
            .js
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.js doesn't exist
            .json
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.json doesn't exist
            .node
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.node doesn't exist
        E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3 doesn't exist
      E:\Work\finsemble-mono\packages\node_modules doesn't exist or is not a directory      
      looking for modules in E:\Work\finsemble-mono\node_modules
        single file module
          using description file: E:\Work\finsemble-mono\package.json (relative path: ./node_modules/@aws-sdk/client-s3)
            no extension
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3 doesn't exist
            .ts
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.ts doesn't exist       
            .js
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.js doesn't exist       
            .json
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.json doesn't exist     
            .node
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.node doesn't exist     
        E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3 doesn't exist
      E:\Work\node_modules doesn't exist or is not a directory
      E:\node_modules doesn't exist or is not a directory
 @ ./node_modules/unzipper/unzip.js 5:0-36
 @ ./src/main/ExternalApplicationManager.ts 4:0-32 70:27-43
 @ ./src/main/Main.ts 4:0-74 120:43-69
 @ ./src/app.ts 12:0-35 100:17-21

ERROR in E:\Work\finsemble-mono\packages\electron-adapter\src\main\ExternalApplicationManager.ts
33:20-40
[tsl] ERROR in E:\Work\finsemble-mono\packages\electron-adapter\src\main\ExternalApplicationManager.ts(33,21)
      TS2307: Cannot find module '@aws-sdk/client-s3' or its corresponding type declarations.
ts-loader-default_04bb2ec108ef140d

webpack 5.91.0 compiled with 2 errors and 2 warnings in 18167 ms
aashimawadhwaa commented 1 month ago

@MaksimDedov plus one facing the same issue.

revolunet commented 1 month ago

same here

ZJONSSON commented 1 month ago

To keep node-unzipper super small, a decision was made to not include optional third party sdks as a part of the library itself. unzipper has plenty of users that do not require the s3 features and it would be very inefficient to force them to do so (example: https://www.npmjs.com/package/exceljs)

If you want to use the aws_s3 you will have to ensure that the sdk is installed, i.e. npm install unzipper @aws-sdk/client-s3, perhaps the right thing to do is clarify this in the README

ZJONSSON commented 1 month ago

@MaksimDedov can you confirm the build error only occurs because you are actually use the aws_v3 in the build process?

MaksimDedov commented 1 month ago

@ZJONSSON In my code I use unzipper in the next way and only once:

const unzipperStream = unzipper.Extract({ path: downloadAsset.assetFolder });
fileStream.pipe(unzipperStream);

I don't call any other functions. The downloadAsset.assetFolder is the path to local folder on machine, fileStream is Node.js transform stream. As I understood the Extract uses aws_v3 somewhere under the hood in ./node_modules/unzipper/lib/Open/index.js 97:52-81

The build error occurs if I don't add the aws_v3 in my package.json near unzipper

ZJONSSON commented 1 month ago

Thank that is very helpful. This seems to be a typescript/ts-loader compilation error not a runtime error. Seems like the appropriate fix here is a simple @ts-ignore.

ZJONSSON commented 1 month ago

@MaksimDedov can you try with unzipper@0.12.3 that was just published?

MaksimDedov commented 1 month ago

unfortunately I see the same issue for unzipper@0.12.3:

ERROR in ./node_modules/unzipper/lib/Open/index.js 98:52-81
Module not found: Error: Can't resolve '@aws-sdk/client-s3' in 'E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\Open'
resolve '@aws-sdk/client-s3' in 'E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\Open'
  Parsed request is a module
  using description file: E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\package.json (relative path: ./lib/Open)
    resolve as module
      E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\Open\node_modules doesn't exist or is not a directory
      E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\lib\node_modules doesn't exist or is not a directory
      looking for modules in E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules
        single file module
          using description file: E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\package.json (relative path: ./node_modules/@aws-sdk/client-s3)
            no extension
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3 doesn't exist
            .ts
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.ts doesn't exist
            .js
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.js doesn't exist
            .json
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.json doesn't exist
            .node
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3.node doesn't exist
        E:\Work\finsemble-mono\packages\electron-adapter\node_modules\unzipper\node_modules\@aws-sdk\client-s3 doesn't exist
      E:\Work\finsemble-mono\packages\electron-adapter\node_modules\node_modules doesn't exist or is not a directory
      looking for modules in E:\Work\finsemble-mono\packages\electron-adapter\node_modules
        single file module
          using description file: E:\Work\finsemble-mono\packages\electron-adapter\package.json (relative path: ./node_modules/@aws-sdk/client-s3)
            no extension
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3 doesn't exist
            .ts
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.ts doesn't exist
            .js
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.js doesn't exist
            .json
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.json doesn't exist
            .node
              E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3.node doesn't exist
        E:\Work\finsemble-mono\packages\electron-adapter\node_modules\@aws-sdk\client-s3 doesn't exist
      E:\Work\finsemble-mono\packages\node_modules doesn't exist or is not a directory
      looking for modules in E:\Work\finsemble-mono\node_modules
        single file module
          using description file: E:\Work\finsemble-mono\package.json (relative path: ./node_modules/@aws-sdk/client-s3)
            no extension
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3 doesn't exist
            .ts
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.ts doesn't exist
            .js
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.js doesn't exist
            .json
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.json doesn't exist
            .node
              E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3.node doesn't exist
        E:\Work\finsemble-mono\node_modules\@aws-sdk\client-s3 doesn't exist
      E:\Work\node_modules doesn't exist or is not a directory
      E:\node_modules doesn't exist or is not a directory
 @ ./node_modules/unzipper/unzip.js 5:0-36
 @ ./src/main/ExternalApplicationManager.ts 4:0-32 70:27-43
 @ ./src/main/Main.ts 4:0-74 120:43-69
 @ ./src/exports.ts 31:24-46

webpack 5.91.0 compiled with 1 error and 2 warnings in 19696 ms
Juknum commented 1 month ago

Same error on my side ^^"

"unzipper": "^0.12.3"

Stack trace from Next:

Module not found: Can't resolve '@aws-sdk/client-s3'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/unzipper/unzip.js
./src/server/actions/files.ts
./src/server/data/mods.ts
./node_modules/next/dist/build/webpack/loaders/next-flight-action-entry-loader.js?actions=%5B%5B%22E%3A%5C%5CGitHub%5C%5Cfaithful%5C%5Cwebsite%5C%5Csrc%5C%5Cserver%5C%5Cdata%5C%5Cmods.ts%22%2C%5B%22getModsWithVersions%22%2C%22deleteMod%22%2C%22updateModPicture%22%2C%22voidMods%22%2C%22getMods%22%2C%22getModsFromIds%22%2C%22getModWithModVersions%22%2C%22updateMod%22%2C%22createMod%22%2C%22modHasUnknownVersion%22%5D%5D%2C%5B%22E%3A%5C%5CGitHub%5C%5Cfaithful%5C%5Cwebsite%5C%5Csrc%5C%5Cserver%5C%5Cdata%5C%5Cmods-version.ts%22%2C%5B%22getModVersionProgression%22%2C%22createModVersion%22%2C%22getSupportedMinecraftVersions%22%2C%22removeModpackFromModVersion%22%2C%22getModsVersionsFromResources%22%2C%22updateModVersion%22%2C%22deleteModVersion%22%2C%22getModVersionsWithModpacks%22%2C%22getNumberOfTextureFromModVersion%22%2C%22getModVersions%22%2C%22addModVersionsFromJAR%22%5D%5D%5D&__client_imported__=true!
 GET / 500 in 1049ms
 GET /_next/static/webpack/a1548000c16fc3ef.webpack.hot-update.json 500 in 1028ms
 ⚠ Fast Refresh had to perform a full reload due to a runtime error.

And the only thing I'm doing with unzipper is the following:

const bytes   = await jar.arrayBuffer(); // where jar is a File
const buffer  = Buffer.from(bytes);
const archive = await unzipper.Open.buffer(buffer);
kelima-jd commented 1 month ago

Similar issue when using "unzipper": "^0.12.3" and NextJS

> next build

  ▲ Next.js 14.2.5
  - Environments: .env

   Creating an optimized production build ...
Failed to compile.

./node_modules/unzipper/lib/Open/index.js
Module not found: Can't resolve '@aws-sdk/client-s3'

https://nextjs.org/docs/messages/module-not-found

Temporary workaround: Add client-s3 to devDependencies

npm i -D @aws-sdk/client-s3@latest