search

serverless-heaven / serverless-webpack

Serverless plugin to bundle your lambdas with Webpack
MIT License
1.72k stars 416 forks source link

support symlinked packages #580

Closed pharindoko closed 4 years ago

pharindoko commented 4 years ago

Hello,

it seems like I`m unable to use symlinked packages via lerna. When I start it with sls offline it works. When I try to deploy it always takes the npm - version of my (monorepo) package instead of the local version

This is a Feature Proposal

Description

For feature proposals: detect if a package is symlinked and copy/attach this to the node_modules folder for webpack.

Similar or dependent issue(s):

Additional Data

pharindoko commented 4 years ago

Any workaround or help is appreciated too. I cannot use local symlinked packages added by "lerna add" command.

pharindoko commented 4 years ago

Anyone here that found a workaround for that issue ?

miguel-a-calles-mba commented 4 years ago

What commands did you use to the symlink?

pharindoko commented 4 years ago

Hey, I have a monorepository managed by lerna (https://github.com/pharindoko/json-serverless). I use this specific command in lerna https://github.com/lerna/lerna/tree/master/commands/add

What I want to achieve:

=> library is symlinked in the templates package.json => newest code of library should be used directly out of templates node_modules folder instead of installing last version via npm. Otherwise I can`t test it before I release the new package in npm.

br,

Flo

pharindoko commented 4 years ago

Hey, I have a monorepository managed by lerna (https://github.com/pharindoko/json-serverless). I use this specific command in lerna https://github.com/lerna/lerna/tree/master/commands/add

What I want to achieve:

=> library is symlinked in the templates package.json => newest code of library should be used directly out of templates node_modules folder instead of installing last version via npm. Otherwise I can`t test it before I release the new package in npm.

br,

Flo

@miguel-a-calles-mba does this answer your question ?

pharindoko commented 4 years ago

@miguel-a-calles-mba @HyperBrain

Hey guys, please at least send a short reply. If this will be not supported in future that`s ok - but please let me know.

br,

Flo

miguel-a-calles-mba commented 4 years ago

Sorry for the delay. I was busy meeting a deadline at work and other things.

It seems this might be an issue with the webpack package and not serverless-webpack.

https://github.com/webpack/webpack/issues/1643 https://github.com/webpack/webpack/issues/985 https://github.com/webpack/webpack/issues/1866 https://github.com/webpack/webpack/issues?q=is%3Aissue+is%3Aopen+symlink

Check out some of the suggested solutions and see whether they help.

pharindoko commented 4 years ago

Thanks a lot for this list. I will check it.

I would have expected that the error happens because sls webpack installs all the npm packages again in the .webpack folder and this is why it doesn`t recognize the symlink in the node_modules.

miguel-a-calles-mba commented 4 years ago

You might want to consider this workaround.

https://github.com/serverless-heaven/serverless-webpack/issues/558

ardalann commented 4 years ago

@pharindoko I'm having a similar issue (using Lerna; getting a "Not found" error for symlinked packages when I try to build). Did you find a solution?

pharindoko commented 4 years ago

Hi @ardalann, I didn`t have time to play around again with this.

My issue is described with this line: https://github.com/serverless-heaven/serverless-webpack/blob/8859f1b492c1c97de28e8c4e5cd18044fadfb248/lib/packExternalModules.js#L326 It reinstalls the package.json modules completely new in a not symlinked folder e.g. .webpack and won`t use the symlinked version in the current node_modules folder.

when I run it with sls offline it does indeed work because there it uses the node_modules folder.

mogusbi-motech commented 4 years ago

I've just ran into this issue today. Here's the debug output when it tries and fails to deploy;

Serverless: Invoke webpack:package
Serverless: Fetch dependency graph from /platform/applications/open-banking/package.json
Serverless: Ignoring 1 NPM errors:
Serverless: => missing: @motech-development/api-gateway-handler@^0.0.0, required by @motech-development/open-banking@0.0.0
Serverless: Package lock found - Using locked versions
Serverless: Packing external modules: axios@^0.19.2, @motech-development/api-gateway-handler@^0.0.0, yup@^0.28.3

Error --------------------------------------------------

npm install failed with code 1
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@motech-development%2fapi-gateway-handler - Not found
npm ERR! 404 
npm ERR! 404  '@motech-development/api-gateway-handler@^0.0.0' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 It was specified as a dependency of 'dependencies'
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/mo/.npm/_logs/2020-05-13T14_29_52_591Z-debug.log

One way you can get around this issue (which isn't ideal) is (if you are using the webpack-node-externals package) to include it in the whitelist array so that it just gets bundled in.

ardalann commented 4 years ago

@mogusbi-motech @pharindoko I managed to work around the issue by switching to the file:../package-name notation in my dependencies.

I did not need to add my symlinked module to webpack-node-externals's whitelist. But I did need to use the following setting in serverless.yml to get rid of my dependency's devDependencies which were raising my bundle size way above the 250mb limit:

  webpack:
    packagerOptions:
      scripts:
        - (cd node_modules/package-name && npm prune --production)
pharindoko commented 4 years ago

@ardalann

@mogusbi-motech @pharindoko I managed to work around the issue by switching to the file:../package-name notation in my dependencies.

I did not need to add my symlinked module to webpack-node-externals's whitelist. But I did need to use the following setting in serverless.yml to get rid of my dependency's devDependencies which were raising my bundle size way above the 250mb limit:

  webpack:
    packagerOptions:
      scripts:
        - (cd node_modules/package-name && npm prune --production)

Yeah. The documentation talks about changing to file: ../package-name but then it`s not symlinked.

using @mogusbi-motech`s approach is what I would have tried now as well. combining webpack-node-externals and npm prune --production might be a nice option.

ed-sparkes commented 4 years ago

Hi,

I am having a similar issue with yarn.

trying to include a local symlinked package called "@my-project/logging" 

Serverless: Invoke webpack:package

  Error --------------------------------------------------

  yarn list --depth=1 --json --production failed with code 1
  {"type":"error","data":"An unexpected error occurred: \"https://registry.yarnpkg.com/@my-project%2flogging: Not found\"."}

any thoughts welcome

pharindoko commented 4 years ago

I give up on this issue. This issue has been adressed several times in other tickets and nothing happened. I kicked out sls-webpack - a separate npm run build and sls deploy fit as well.