Lighten the dependency load

[davemo]

Given the discussion in previous issues around the weight of linemans dependency graph when installed locally, I'm wondering if it's possible to consider again how we can reduce this to the minimum.

Below is a listing of my node_modules folder disk usage without lineman, which is 54mb lighter than it would be with lineman, for nearly the same workflow.

Discuss. :)

$ du -hd1 node_modules/
 12K    node_modules//.bin
500K    node_modules//coffee-script
340K    node_modules//glob
5.4M    node_modules//grunt
 68K    node_modules//grunt-batman-templates
380K    node_modules//grunt-concat-sourcemap
124K    node_modules//grunt-contrib-clean
1.0M    node_modules//grunt-contrib-coffee
 28K    node_modules//grunt-contrib-copy
420K    node_modules//grunt-contrib-cssmin
 24M    node_modules//grunt-contrib-imagemin
1.4M    node_modules//grunt-contrib-uglify
740K    node_modules//grunt-datauri-variables
2.9M    node_modules//grunt-http
284K    node_modules//grunt-newer
132K    node_modules//grunt-parallel
108K    node_modules//grunt-rails-asset-digest
7.6M    node_modules//grunt-sass
164K    node_modules//semver
 72K    node_modules//underscore
 46M    node_modules/

[jraede]

Isn't this more of an NPM dependency resolution issue than a Lineman issue at this point? — Sent from Mailbox for iPhone

On Thu, Feb 6, 2014 at 1:16 PM, David Mosher notifications@github.com wrote:

Given the discussion in previous issues around the weight of linemans dependency graph when installed locally, I'm wondering if it's possible to consider again how we can reduce this to the minimum. Below is a listing of my node_modules folder disk usage without lineman, which is 54mb lighter than it would be with lineman, for nearly the same workflow. Discuss. :)

$ du -hd1 node_modules/
 12K  node_modules//.bin
500K  node_modules//coffee-script
340K  node_modules//glob
5.4M  node_modules//grunt
 68K  node_modules//grunt-batman-templates
380K  node_modules//grunt-concat-sourcemap
124K  node_modules//grunt-contrib-clean
1.0M  node_modules//grunt-contrib-coffee
 28K  node_modules//grunt-contrib-copy
420K  node_modules//grunt-contrib-cssmin
 24M  node_modules//grunt-contrib-imagemin
1.4M  node_modules//grunt-contrib-uglify
740K  node_modules//grunt-datauri-variables
2.9M  node_modules//grunt-http
284K  node_modules//grunt-newer
132K  node_modules//grunt-parallel
108K  node_modules//grunt-rails-asset-digest
7.6M  node_modules//grunt-sass
164K  node_modules//semver
 72K  node_modules//underscore
 46M  node_modules/

---

Reply to this email directly or view it on GitHub: https://github.com/linemanjs/lineman/issues/201

[davemo]

Here's a top level listing of the lineman projects node_modules folder:

$ du -hd1 node_modules/
 28K    node_modules//.bin
404K    node_modules//coffee-script
 72K    node_modules//commander
 40K    node_modules//config-extend
5.0M    node_modules//express
5.2M    node_modules//grunt
1020K   node_modules//grunt-asset-fingerprint
488K    node_modules//grunt-concat-sourcemap
124K    node_modules//grunt-contrib-clean
188K    node_modules//grunt-contrib-coffee
136K    node_modules//grunt-contrib-copy
284K    node_modules//grunt-contrib-cssmin
1.7M    node_modules//grunt-contrib-handlebars
1.4M    node_modules//grunt-contrib-jshint
768K    node_modules//grunt-contrib-jst
9.0M    node_modules//grunt-contrib-less
 68K    node_modules//grunt-contrib-sass
1.4M    node_modules//grunt-contrib-uglify
1.2M    node_modules//grunt-release
524K    node_modules//grunt-watch-nospawn
1.2M    node_modules//http-proxy
300K    node_modules//lodash
152K    node_modules//resolve
144K    node_modules//semver
 23M    node_modules//testem
616K    node_modules//underscore.string
852K    node_modules//watch_r-structr-lock
 56M    node_modules/

[jraede]

Right but given Lineman's dependencies in order to do what it does, it needs all of those in one way or another. The only option would be to somehow detect the presence of those packages in the project and not download them for lineman, but that would fall under the responsibilities of NPM would it not?

[jayharris]

Why do you feel that Lineman needs to do all of these in one way or another? To me, these are just included as a part of the Lineman default configuration as a config/plugin, included to help new developers ramp up. There isn't any technical dependency, just a convenience to aid discovery.

The default Archetype & configuration doesn't use Handlebars, nor a Sass, nor AssetFingerprinting. They are there for discovery, and their 3M downloaded even if you never use them. If you do enable Sass, you're probably not going to use Less, which is another 9M.

As we are building larger and larger applications in Lineman, this is becoming more of an issue. In Azure, there's limited temporary space (where the application is built / compiled before deployment), and I assume there is the came issue in Heroku. We're starting to run up against it, and we too have had to pull a few applications out of lineman to strip down the footprint.

[jraede]

@jayharris I see what you're saying - it may make sense to strip everything out and let people define their own dist, common, dev, etc tasks based on what they want to install locally. Maybe some "common workflows" with lists of dependencies that the user would have to install themselves.

[jayharris]

I like the discovery aspects, and would like to keep them around for just that purpose. However, I still propose stripping everything out, extracting it to a standalone plugin. Lineman core wouldn't reference these plugins itself; it would be in the Archetype's package.json. This way they are still there for discovery, but npm uninstall X --save would strip them out entirely.

Also, with this model, we can set the Archetype's default package.json to have the build plugins (Less, Sass, Underscore, CSSmin) all be Production dependencies, and the test (run) plugins (Express, http-proxy, watch_r, testem) be Dev dependencies. This way, when Heroku or Azure builds the site, it will pull down only the dependencies necessary to build and deploy the app, saving more space (over 30M) in that tiny Temp footprint.

[jasonkarns]

An option to consider is sorta like a scaffolding command. If lineman new ran through some prompts, we could add the appropriate lineman-* plugins to the package.json. It could prompt for JS/Coffee/(other as supported), CSS/Sass/LESS/Stylus..., Handlebars/Underscore/Jade/etc...​