serverless-heaven / serverless-webpack

Serverless plugin to bundle your lambdas with Webpack
MIT License
1.72k stars 416 forks source link

.webpack/node_modules are not part of the serverless zip anymore. #407

Closed a-marcel closed 6 years ago

a-marcel commented 6 years ago

This is a (Bug Report / Feature Proposal)

Description

For bug reports:

const sharpTarball = path.resolve( dirname, lambda-sharp/sharp.tar.gz ) const webpackDir = path.resolve(dirname, '.webpack/')

function ExtractTarballPlugin (archive, to) { return { apply: (compiler) => { compiler.plugin('emit', (compilation, callback) => { decompress(path.resolve(archive), path.resolve(to)) .then(() => callback()) .catch(error => console.error('Unable to extract archive ', archive, to, error.stack)) }) }, } }

module.exports = { entry: slsw.lib.entries, target: 'node', mode: slsw.lib.webpack.isLocal ? "development" : "production", module: { rules: [ { test: /.js$/, include: __dirname, exclude: /node_modules/, use: [ 'babel-loader' ] }, { test: /.json$/, use: [ 'json-loader' ] } ] }, output: { libraryTarget: 'commonjs', path: path.resolve(dirname, '.webpack'), filename: '[name].js', // this should match the first part of function handler in serverless.yml }, externals: ['aws-sdk', 'sharp'], plugins: [ // new CleanWebpackPlugin(path.join(dirname, '.webpack/node_modules')), // new webpack.optimize.OccurenceOrderPlugin(), // new webpack.optimize.DedupePlugin(), // new webpack.optimize.UglifyJsPlugin({ minimize: true, sourceMap: false, warnings: false }), new ExtractTarballPlugin(sharpTarball, webpackDir), ], }



## Additional Data

* ***Serverless-Webpack Version you're using***:
5.1.5
* ***Webpack version you're using***:
4.12.0
* ***Serverless Framework Version you're using***:
1.27.3
* ***Operating System***:
MacOs
* ***Stack Trace (if available)***:
HyperBrain commented 6 years ago

@marcelalburg Can you try to use nodeExternals and not only import it?

// webpack.config.js
...
  externals: [nodeExternals()],
  ...

This will make sure that everything is packaged correctly depending on the dependencies of the compiled code. Additionally, set custom: webpack: includeModules: true in your serverless.yml.

If you need to enforce the packaging of sharp, you should use forceInclude as follows:

# serverless.yml
custom:
  webpack:
    includeModules:
      forceInclude:
        - sharp
a-marcel commented 6 years ago

Hello, thanks for your fast answer.

sharp is not part of the node_modules in the beginning and not in the package.json dependencies. Webpack will copy and extract the lambda-sharp/sharp.tar.gz file to the .webpack folder after build.

If i put nodeExternals() to externals, webpack will not compile because sharp is not there.

I even tried to put you mentioned serverless.yml changes, but this is not working too.

After the webpack build is finished the attached folder structure are inside the .webpack folder. My expectation was/is, that exactly this is part of the serverless .zip file.

Thanks Marcel

.webpack/
.webpack//node_modules
.webpack//node_modules/simple-swizzle
.webpack//node_modules/.bin
.webpack//node_modules/chownr
.webpack//node_modules/color-string
.webpack//node_modules/detect-libc
.webpack//node_modules/detect-libc/bin
.webpack//node_modules/detect-libc/lib
.webpack//node_modules/once
.webpack//node_modules/nan
.webpack//node_modules/nan/tools
.webpack//node_modules/nan/doc
.webpack//node_modules/safe-buffer
.webpack//node_modules/color
.webpack//node_modules/isurl
.webpack//node_modules/color-name
.webpack//node_modules/config-chain
.webpack//node_modules/config-chain/test
.webpack//node_modules/has-to-string-tag-x
.webpack//node_modules/has-to-string-tag-x/lib
.webpack//node_modules/decompress-response
.webpack//node_modules/simple-get
.webpack//node_modules/proto-list
.webpack//node_modules/proto-list/test
.webpack//node_modules/pify
.webpack//node_modules/has-symbol-support-x
.webpack//node_modules/has-symbol-support-x/tests
.webpack//node_modules/has-symbol-support-x/tests/spec
.webpack//node_modules/has-symbol-support-x/lib
.webpack//node_modules/minimist
.webpack//node_modules/minimist/test
.webpack//node_modules/minimist/example
.webpack//node_modules/mimic-response
.webpack//node_modules/npm-conf
.webpack//node_modules/npm-conf/lib
.webpack//node_modules/color-convert
.webpack//node_modules/caw
.webpack//node_modules/minizlib
.webpack//node_modules/semver
.webpack//node_modules/semver/bin
.webpack//node_modules/ini
.webpack//node_modules/get-proxy
.webpack//node_modules/simple-concat
.webpack//node_modules/simple-concat/test
.webpack//node_modules/is-arrayish
.webpack//node_modules/is-object
.webpack//node_modules/is-object/test
.webpack//node_modules/minipass
.webpack//node_modules/yallist
.webpack//node_modules/wrappy
.webpack//node_modules/tar
.webpack//node_modules/tar/lib
.webpack//node_modules/tunnel-agent
.webpack//node_modules/sharp
.webpack//node_modules/sharp/docs
.webpack//node_modules/sharp/lib
.webpack//node_modules/sharp/lib/icc
.webpack//node_modules/sharp/build
.webpack//node_modules/sharp/build/Release
.webpack//node_modules/sharp/build/Release/obj.target
.webpack//node_modules/sharp/build/Release/obj.target/sharp
.webpack//node_modules/sharp/build/Release/obj.target/sharp/src
.webpack//node_modules/sharp/build/Release/.deps
.webpack//node_modules/sharp/build/Release/.deps/Release
.webpack//node_modules/sharp/build/Release/.deps/Release/obj.target
.webpack//node_modules/sharp/build/Release/.deps/Release/obj.target/sharp
.webpack//node_modules/sharp/build/Release/.deps/Release/obj.target/sharp/src
.webpack//node_modules/sharp/vendor
.webpack//node_modules/sharp/vendor/include
.webpack//node_modules/sharp/vendor/include/libpng16
.webpack//node_modules/sharp/vendor/include/libxml2
.webpack//node_modules/sharp/vendor/include/libxml2/libxml
.webpack//node_modules/sharp/vendor/include/freetype2
.webpack//node_modules/sharp/vendor/include/freetype2/freetype
.webpack//node_modules/sharp/vendor/include/freetype2/freetype/config
.webpack//node_modules/sharp/vendor/include/pixman-1
.webpack//node_modules/sharp/vendor/include/libexif
.webpack//node_modules/sharp/vendor/include/harfbuzz
.webpack//node_modules/sharp/vendor/include/vips
.webpack//node_modules/sharp/vendor/include/librsvg-2.0
.webpack//node_modules/sharp/vendor/include/librsvg-2.0/librsvg
.webpack//node_modules/sharp/vendor/include/gdk-pixbuf-2.0
.webpack//node_modules/sharp/vendor/include/gdk-pixbuf-2.0/gdk-pixbuf
.webpack//node_modules/sharp/vendor/include/cairo
.webpack//node_modules/sharp/vendor/include/webp
.webpack//node_modules/sharp/vendor/include/gio-unix-2.0
.webpack//node_modules/sharp/vendor/include/gio-unix-2.0/gio
.webpack//node_modules/sharp/vendor/include/fontconfig
.webpack//node_modules/sharp/vendor/include/libgsf-1
.webpack//node_modules/sharp/vendor/include/libgsf-1/gsf
.webpack//node_modules/sharp/vendor/include/glib-2.0
.webpack//node_modules/sharp/vendor/include/glib-2.0/gobject
.webpack//node_modules/sharp/vendor/include/glib-2.0/glib
.webpack//node_modules/sharp/vendor/include/glib-2.0/glib/deprecated
.webpack//node_modules/sharp/vendor/include/glib-2.0/gio
.webpack//node_modules/sharp/vendor/include/pango-1.0
.webpack//node_modules/sharp/vendor/include/pango-1.0/pango
.webpack//node_modules/sharp/vendor/include/orc-0.4
.webpack//node_modules/sharp/vendor/include/orc-0.4/orc
.webpack//node_modules/sharp/vendor/include/orc-0.4/orc-test
.webpack//node_modules/sharp/vendor/include/libcroco-0.6
.webpack//node_modules/sharp/vendor/include/libcroco-0.6/libcroco
.webpack//node_modules/sharp/vendor/lib
.webpack//node_modules/sharp/vendor/lib/cmake
.webpack//node_modules/sharp/vendor/lib/cmake/libxml2
.webpack//node_modules/sharp/vendor/lib/gdk-pixbuf-2.0
.webpack//node_modules/sharp/vendor/lib/gdk-pixbuf-2.0/2.10.0
.webpack//node_modules/sharp/vendor/lib/libffi-3.2.1
.webpack//node_modules/sharp/vendor/lib/libffi-3.2.1/include
.webpack//node_modules/sharp/vendor/lib/glib-2.0
.webpack//node_modules/sharp/vendor/lib/glib-2.0/include
.webpack//node_modules/sharp/vendor/lib/gio
.webpack//node_modules/sharp/vendor/lib/gio/modules
.webpack//node_modules/sharp/src
.webpack//node_modules/sharp/src/libvips
.webpack//node_modules/sharp/src/libvips/cplusplus
.webpack//node_modules/url-to-options
.webpack//node_modules/mkdirp
.webpack//node_modules/mkdirp/test
.webpack//node_modules/mkdirp/bin
.webpack//node_modules/mkdirp/examples
HyperBrain commented 6 years ago

Not exactly. If you use individual packaging, the function zips are created from .webpack/<function_name>/*, otherwise from .webpack/service/*.

.webpack/node_modules is only the common first-stage module collection and that will be copied into the final folder depending on the actually detected dependencies.

If you copy the sharp module you have to copy it to the "final" folder, not .webpack/node_modules. I think you can also use a custom script to do that, because then the CWD is already the right dir.

So, add the following to your serverless.yml:

# serverless.yml
custom:
  webpack:
    packagerOptions:
      scripts:
        - tar xzf lambda-sharp/sharp.tar.gz ....     # Just a sample. Should be the extraction command

This will make the extraction the very last step after the package contents have been fully generated.

a-marcel commented 6 years ago

Hello,

i created an example github repo for this: https://github.com/marcelalburg/serverless-webpack-sharp. Maybe you can take a look. Even with your mentioned changes, after i entered "npm run severless -- package", the final zip package is not containing the node_modules.

Thx

a-marcel commented 6 years ago

Hello, it worked as you mentioned now. I just had a small typo in my packagerOptions script. Thx a lot.

a-marcel commented 6 years ago

Ok, i found a better way (like before). In the new version the path is not the same anymore.

instead of .webpack it's now .webpack/service

i just need to extract my files into this folder and everything works as before.

HyperBrain commented 6 years ago

@marcelalburg Glad to hear it works now.