Why so big? What is usually needed?

sdarnell commented 7 years ago

I was going to add a comment to one of the other issues relating to insufficient documentation about this module, but I figured a separate issue would be better based around its size.

Using the new bundle visualizer in meteor 1.5 (which is great BTW), I discovered that meteor-node-stubs was taking up 600k of my production bundle, and there is very little info about why one might need it, whether any existing meteor (core) packages rely on it etc. It probably shouldn't be included by default given it is so big, or split up into handy-but-small-stuff, and other-stuff-but-big.

For reference, here an annotated diagram showing it relative to some other big things in our app:

Interestingly, I uninstalled meteor-node-modules and confirmed things continued to work, which they seemed to (except needing assert module for the visualizer). Then re-installed meteor-node-modules to give more info about which sub sections were taking most space. But the size changed! Presumably due to the optimisations performed by the newer NPM?

Now it takes a measly (cough): 309kb, of which:

eliptic takes 80kb
bn.js takes 43kb
asn1 takes 24kb
readable-stream takes 20kb
buffer takes 20kb
browserify-aes takes 15kb
etc.

I can understand general stuff like the streams and buffer implementations, but the encryption/crypto stuff seems specialised and large. Maybe it should be separated?

benjamn commented 7 years ago

Something in your client code is likely calling require("crypto"), and may break if that no longer works. Worth finding that call site and figuring out why it's needed.

Only modules you're actually using get bundled, so merely installing meteor-node-stubs without using it adds nothing to your bundle. However, crypto and its dependencies are pretty big!

sdarnell commented 7 years ago

Thanks. I certainly don't have any explicit references to crypto (either client or server). Any suggestions for identifying what is causing extras to be included? (e.g. is there any logging that can be turned on for this sort of thing).

Looking at indirect node_module references (ignoring meteor packages for now), as the server are client modules are not separated, it is hard to tell which are the live references for the client. Matters are confused by things like the uuid package for example, which unsurprisingly has a require('crypo') in uuid/lib/rng.js but uses a package.json mapping:

  "browser": {
    "./lib/rng.js": "./lib/rng-browser.js"
  },

and rng-browser.js detects global.crypto || global.msCrypto and falls back to Math.random().

Does meteor honour the package.js browser mappings?

Another tricky package seems to be zone.js, which contains a zone-node.js (which I don't think is included) but has:

try {
    crypto = require('crypto');
}
catch (err) {
}

sdarnell commented 7 years ago

By the way, I decided to look at the bundle generated by meteor --production, and the crypto references were:

atmosphere package 'random' which includes a Npm.require("crypto")
the 'random' npm package references a global 'crypto' (from lib/rng-browser.js) so that's good! But there were no other references. Surely the atmosphere 'random' (an MDG package) doesn't trigger the pulling in of meteor_node_stubs crypto stuff?

jlichti commented 7 years ago

I was seeing the same with the bundle visualizer when experimenting to reduce bundle size.

The 315kb for meteor-node-stubs was my largest slice so wanted to figure out what was triggering. I commented out my routes file to just look at the dependencies.

For me the culprit was the okgrow:analytics package. My bundle size is 696kb without that package and 1.18MB with.

RobertLowe commented 6 years ago

@benjamn yeah this is pretty huge, I just yak shaved my app down to 650k (minified) to 1.5mb (unminified)

In my case I was pulling socket-io client from NPM, that saved about 300k as it was using crypto , removing stubs saved about 400k-ish. I also savagely hack a few core and community packages to remove jQuery as dependancy that save another hundred or so, now I load both of those from public CDNs. This resolved the double loading of jQuery and added the CDN bonus.

I hacked:

twbs/bootstrap
useraccounts/core
softwarerero/accounts-t9n
blaze
kadira/blaze-layout
accounts-ui-unstyled
amplify

and removed the jQuery deps, they seem to be working fine.

I also added:

jquery-cdn
socket.io-cdn

As replacements.

Additionally, I hacked softwarerero/accounts-t9n and remove another 40-50kb or so as for now I only need en.fr translations

Things feel fast

etyp commented 5 years ago

@sdarnell did you ever figure out the source of your crypto import?

etyp commented 5 years ago

@benjamn I'm wondering if there's any easy way to track which files trigger meteor-node-stubs to be used on the client. Is there some step in the build process to hook into?

As of right now we're limited to untangling dependencies and re-running the visualizer each time to see what files trigger it, but it's pretty tedious as you can imagine. Even looking at the .stats.json it's not totally clear

sdarnell commented 5 years ago

@etyp it was a while ago and I don't recall with certainty, but I think I fixed it by replacing okgrow:analytics with a more stream-lined inclusion of Google analytics.

sdarnell commented 5 years ago

It certainly would be nice to have some way of outputting the npm dependencies in a similar way to the merteor packages (meteor list -tree added with https://github.com/meteor/meteor/pull/8936).

RobertLowe commented 5 years ago

@sdarnell have a look at your package-lock.json and look at your imports, but also just scroll through you app/app.js in development mode

filipenevola commented 4 years ago

I'm closing this as we don't have activity for more than a year.

meteor / node-stubs

Why so big? What is usually needed? #6