Recent increase in deployed slug size

[jtgeibel]

Looking at some recent deploys, the compiled slug size initially dropped a bit and then grew in size.

• Production
  • 09-19 (~1 month of merges) - 270.9M
  • 09-12 (hotfix) - 156.3M
  • 09-09 (hotfix) - 156.1M
  • 08-23 - 156.2M
  • 08-17 - 212.8M
  • 08-11 - 208.9M
  • 07-27 - 205.8M
• Staging
  • Shows a similar jump in size from 152.3M in 0eb1998c to 270.6M in 8fb107a6.
  • https://github.com/rust-lang/crates.io/compare/0eb1998...8fb107a6

From what I recall, a size of about 200M has been typical until fairly recently.

I originally started investigating this because in #2806 I saw a decrease to ~90M in my own staging area. Strangely, I have not seen the same decrease when deploying to the official staging area.

My first guess is that devDependencies are being included in the slug. When I run npm install locally (node 14.9.0, npm 6.14.8) I get a diff that marks more dependencies as "dev": true. However that build on staging still resulted in a 265.5M slug. (This doesn't rule out the theory that dev dependencies are being included, just that it doesn't appear to be due to the diff between my local setup and the PRs made by Renovate.)

The only other interesting thing I see in that comparison is the change from ember-percy 1.6.0 to @percy/ember 2.1.3 in 63875ef. This introduces a lot of new dependencies, but again I expect them to be pruned by the buildpack :shrug:.

I don't think this is due to anything in the backend, but it's possible.

cc @Turbo87 @locks

[Turbo87]

The only other interesting thing I see in that comparison is the change from ember-percy 1.6.0 to @percy/ember 2.1.3 in 63875ef. This introduces a lot of new dependencies, but again I expect them to be pruned by the buildpack 🤷.

it does seem to use puppeteer for some reason, which downloads a copy of Chromium, but as you said, if the dev deps are getting pruned correctly then this shouldn't make a difference.

When I run npm install locally (node 14.9.0, npm 6.14.8) I get a diff that marks more dependencies as "dev": true.

this is something that is confusing me quite a bit too. I have tried to read up on the dev flag in the npm docs, but it doesn't seem to consistently behave like the docs are describing it.

I also have to admit that crates.io is the single only project where I'm using npm instead of yarn, and with yarn I've never experienced such inconsistency issues.

[jtgeibel]

Okay, I've tracked down the difference in the two environments. It looks like we have NPM_CONFIG_PRODUCTION=false set in staging. By unsetting this devDependencies should be pruned as expected. (I tried setting it to true, but then the build fails because some devDependencies, like ember-cli, are necessary to build the assets.)

I've opened #2819 to further reduce the slug size (though I expect the improvement to be minimal).

[jtgeibel]

Closing as I've confirmed all environments now produce a slug ~90M in size.