AnomalyInnovations / serverless-bundle

Optimized packages for ES6 and TypeScript Node.js Lambda functions without any configuration.
https://serverless-stack.com/chapters/package-lambdas-with-serverless-bundle.html
MIT License
536 stars 157 forks source link

Remove _warmup from the bundle output (serverless-plugin-warmup plugin) #66

Open patrykkrawczyk opened 4 years ago

patrykkrawczyk commented 4 years ago

Hi,

I've created a project based on this repository https://github.com/AnomalyInnovations/serverless-nodejs-starter

Then I've added serverless-plugin-warmup, to do so the only thing I did is:

Ran:
npm install --save-dev serverless-plugin-warmup

and

Updated my serverless.yml to include those lines:

custom:
  warmup:
    enabled: true

plugins:
  - serverless-bundle # Package our functions with Webpack
  - serverless-offline
  - serverless-dotenv-plugin # Load .env as environment variables
  - serverless-plugin-warmup

Now when I do: sls package

I have a zip file in .serverless directory and it contains _warmup directory with index.js This is strange since I believe there is a functionality to remove this directory baked into serverless-bundle. Unfortunately it does not work :(

jayair commented 4 years ago

Hmmm we don't remove the warmup files. We only ignore them for ESLint and Babel, etc. We simply move it over to the package directory.

If we removed them, then the warmup plugin won't work right?

patrykkrawczyk commented 4 years ago

I believe _warmup files is the function code of the Lambda function that is calling a function that needs to be warmed up. It seems that when using warmup plugin after packaging, we get two functions:

  1. a function that warms up other function
  2. a function that is being called by the function 1 Function 2 is put into folder _warmup, and function 1 does not need this folder to work.
jayair commented 4 years ago

@patrykkrawczyk Can you try the same setup in your post but disable the bundle plugin? And see if you see something different?

patrykkrawczyk commented 4 years ago

with serverless-bundle commented out in serverless.yml I get a zip package with all contents of my project inside, along with _warmup folder @jayair

jayair commented 4 years ago

Just to double check, is that the same as if you are using the bundle plugin?

patrykkrawczyk commented 4 years ago

if im using bundle plugin:

without bundle plugin:

jayair commented 4 years ago

I'm a little confused here. Can you tell me what the issue is?

patrykkrawczyk commented 4 years ago

When using warmup plugin together with bundle plugin, after you package your lambda function you will have a zip file in .serverless directory. Within this zip, you will actually have two functions instead of one. If your function name is hello, then within your hello.zip you will have code for the hello function and for the warmup function. So it's basically uploading two functions within a single zip file, bloating up code size. This hello.zip should only have code of the hello function. In .serverless directory, there will also be a warmUpPlugin.zip which holds only the warmup function code, which will be called to warmup the hello function.

patrykkrawczyk commented 4 years ago
\---.serverless
    \---hello.zip
    |   |   hello.js
    |   \---_warmup
    |           index.js
    |
    \---warmUpPlugin.zip
        \---_warmup
                index.js

Notice how the _warmup directory is duplicated in hello.zip

jayair commented 4 years ago

Hmm if I run the steps in your original post with the bundle plugin I get this:

Screen Shot 2020-04-18 at 10 07 25 PM

And with the bundle plugin commented out, I get this:

Screen Shot 2020-04-18 at 10 06 18 PM

The _warmup dir is there in both cases. I don't really see what you are saying.

patrykkrawczyk commented 4 years ago

Thats the issue. Let's focus on the first case. Why is there _warmup directory in hello.zip? It's not used for anything and just increases zip size. hello.zip should only contain handler.js and handler.js.map

For warming up there's another zip called warmUpPlugin.zip.

jayair commented 4 years ago

But from my quick test, this happens with or without the bundle plugin. So it doesn't seem related?

patrykkrawczyk commented 4 years ago

bundle plugin is removing all other files except for the _warmup directory

jayair commented 4 years ago

I don't have a ton of experience with the warmup plugin. But wouldn't it make more sense to tell them not to add that file in the package?

patrykkrawczyk commented 4 years ago

I think during webpack build this directory _warmup is being created to build warmUpPlugin.zip. The files then are picked up by serverless-bundle and are added to other zips. I don't think it's possible for them to mitigate this. The warmUpPlugin.zip package still needs to be build but for other functions it should be ignored like all other files are ignored.

jayair commented 4 years ago

Can you provide steps to reproduce this? Because as I said above, even with the bundle plugin disabled, the files are still there.