Client side JS package manager

[tomaszdurka]

https://github.com/twitter/bower I am not sure how it exactly works when it comes to loading/requiring libraries. BUT you can get from there libraries like:

01-json2.js
02-jquery.js
03-underscore.js
04-backbone.js
swfobject.js

And many more also popular like: bootstrap, bolierplate, including less styles etc. They could be used for apps like IdeaHub.

Solutions:

http://jamjs.org/ https://github.com/substack/node-browserify http://gruntjs.com/ http://volojs.org/ https://github.com/spine/hem https://github.com/component/component https://github.com/mattcg/cjsdelivery - CommonJS modules compiler in PHP http://browserify.org/ https://github.com/google/module-server - Serving CommonJS/AMD modules to browsers in 1 request

[tomaszdurka]

I've checked out bower and the concept behind it is pretty good and actually would easily fit into our framework/philosophy.

Each component.json file should have main files specified which point to required javascript and/or css files (depending on the package). We could install it in some working directory and copy over needed js files to one location and css to different. There is also command bower list --map which would give us parsable json list of libraries with its main files.

Now, this works pretty well for all libraries with component.json specified, but if the file is missing (e.g. Backbone/Underscore) we have a problem. Now the reasons for this file not included are various. Swf object e.g. was simply not updated for almost year and I wander if maybe its owner went missing. Backbone and Underscore have different reasons. I found a pull-request on both of them with with component.json being added, but the owner/maintainer strictly said they will not support it - I quote: "Thanks, but no thanks -- we don't want to add a special configuration file for every new package manager that is released." https://github.com/documentcloud/underscore/pull/884.

This trailed me to maybe using npm, but npm and its package.json is connected more with nodejs and is rather not for front-end stuff which bower is suiting quite well. In the end we could possibly combine this with entries from npm's package.json however that would be quite dirty and unsure.

Back to the topic I think bower suits as very well, however it lacks support for some core libraries. I would really not like to label this with maybe-later :)

[tomaszdurka]

@njam During the weekend I've found / checked out a few other solutions.

One of them is jam. It's yet another package manager for node packages. It helps with so-called frontend packages. There are really helpful commands like compile which after about an hour I could still not get working :) Besides that idea is pretty interesting. It compiles all dependencies from package.json. You can also specify "jam dependencies" - this way splitting them into two types.

There is also library called browserfiy. The library reads main node file and using other library called detective easily gets all "requires". After retrieving them it get all dependencies in a tree-dependency way and include/compile into single file. This was quite easy to run, but it forces us to create this main file to require all dependencey. This could actually be good idea, having js file defining depedencies:

var backbone = require('backbone');
var _ = require('underscore');

In both these cases we probably should use requirejs library which I think is overally good idea.

[tomaszdurka]

Maybe we should also look into Grunt.

[tomaszdurka]

Workaround: https://github.com/cargomedia/CM/pull/280

[njam]

Article on the topic: http://tech.pro/tutorial/1190/package-managers-an-introductory-guide-for-the-uninitiated-front-end-developer

[tomaszdurka]

I think this is closable.

[njam]

@tomaszdurka @vogdb @christopheschwyzer Video - How Instagram uses webpack to load static resources (js, css, images) in modules based on a dependency graph: https://www.youtube.com/watch?v=VkTCL6Nqm6Y Webpack website: https://github.com/webpack/webpack Webpack cookbook: https://github.com/petehunt/webpack-howto

Current loading stats (gzipped):

• JS: before-body.js (6 KB), after-body.js (281 KB), library.js (52 KB), translations.js (1.6 KB)
• CSS: library/all.css (52 KB), vendor/all.css (10 KB)

[stophecom]

fast forwarded through. Interesting...would be worth looking into it from what I understood.

[tomaszdurka]

I will definetely look into that :>

[vogdb]

Thoughts about webpack:

Here and after by the dependencies you should understand not only js files but all the possible web static files, e.g. css, less, image, font and other.

Each js file needs to declare its dependencies.

Let's say we have a component:

var SK_Component_Some = SK_Component_Abstract.extend({
  ...
})

It uses SK_Component_Abstract which is placed in another file. This means that the SK_Component_Some needs to load its SK_Component_Abstract dependency before its execution. Let's do it in the AMD style:

 define('SK_Component_Some', ['relative path to the SK_Component_Abstract'], function(SK_Component_Abstract) {
   var SK_Component_Some = SK_Component_Abstract.extend({
     ...
   })
 });

Above we just described the AMD module. Only when all of our source files are described as AMD or CommonJS modules, we can run webpack on them. Final layout pages, the one that user sees in its browser, are not exceptions to this change and must declare their dependencies too: (CommonJS style)

  var CM_Page_Abstract = require('relative path to the CM_Page_Abstract');

  var SK_Page_Feed = CM_Page_Abstract.extend({
    _class: 'SK_Page_Feed'
  });

But these pages are not that easy. Each page can contain a plenty of components inside itself. So the improved version is:

  var CM_Component_1 = require('relative path to the CM_Component_1');
  var CM_Component_2 = require('relative path to the CM_Component_2');
  //...
  var CM_Component_N = require('relative path to the CM_Component_N');
  var CM_Page_Abstract = require('relative path to the CM_Page_Abstract');

  var SK_Page_Feed = CM_Page_Abstract.extend({
    _class: 'SK_Page_Feed'
  });

And this is not the end. Each page and component might have a style file(css, less). So the correct version is:

  var CM_Component_1 = require('relative path to the CM_Component_1');
  var CM_Component_2 = require('relative path to the CM_Component_2');
  //...
  var CM_Component_N = require('relative path to the CM_Component_N');
  var CM_Page_Abstract = require('relative path to the CM_Page_Abstract');

  require('relative path to the SK_Page_Feed style file');

  var SK_Page_Feed = CM_Page_Abstract.extend({
    _class: 'SK_Page_Feed'
  });

Summary. This adjustment will force us to add a lot of dependencies to our js source files, e.g change a lot in our source files. Besides that these dependencies should be declared as relative paths.

Create a webpack modules and link them with the page templates.

We need to declare what the exact modules we want to see at the end of the webpack's processing. For example, we have the page Feed that is shown when a www.fuboo.com/feed url is browsed. Currently the Feed page doesn't declare such things as <script src='Page/Feed.js'></script> or <link rel='stylesheet' href='Page/Feed/default.css'/> because we have one js and css file that contain all of the js and css of all the the pages including Page/Feed.js and Page/Feed/default.less. But webpack will change it and each page will have its own separate module of static files. To achieve this we need to describe each page's module in the webpack config. In short the config for the only one Feed page would look like this:

module.exports = {
  entry: {
    Feed: "./Page/Feed.js"
  },
  output: {
    filename: "[name].module.js"
  }
};

Above config will generate the file Feed.module.js which would contain all of the static files that are required to show the page.

Config for two pages Feed and About would look like this:

module.exports = {
  entry: {
    Feed: "./Page/Feed.js",
    About: "./Page/About.js"
  },
  output: {
    filename: "[name].module.js"
  }
};

It will generate files Feed.module.js and About.module.js. It is important that these .module.js files contain all of the required js, css and other static files. So now we need to let the Feed and About pages know about these .module.js files. For that we need to put into the page template a script tag. Like this:

<script src='/vendor.module.js'></script>
<script src='/common.module.js'></script>
<script src='/Page/Feed.module.js'></script>
{extends file=$render->getLayoutPath('Page/Abstract/two-column.tpl')}

{block name='content-title'}Feed{/block}
{*
  ...
*}

Summary. This adjustment will force us to change a lot of template files with dependencies to their .module.js files.

Final Summary

Webpack integration requires a lot of changes in the source files. From one point of view I think these changes are useful. It's good to see on what files the page template depends upon. It's good to see what static files are required for js view. From another point of view I think we can avoid these changes if we create a special webpack plugin for our needs. Such plugin will introduce an additional config file where we describe all the dependencies among the static files. Here is the simple draft of it:

module.exports = {
  entry: {
    Feed: {
      dest: "./Page/Feed.module.js",
      dependencies: ["./Page/Abstract.js", "./Page/Feed.js", "./Page/Feed/default.css", ...]
    },
    About: {
      dest: "./Page/About.js",
      dependencies: ["./Page/Abstract.js", "./Page/About.js", "./Page/Feed/default.css", ...]
    }
  }
};

This will let us escape this type of changes:

  var CM_Page_Abstract = require('./Page/Abstract.js');
  require('./Page/Feed/default.css');
  var SK_Page_About = CM_Page_Abstract.extend({//...

Changes in template files can be minimized from this:

<script src='/vendor.module.js'></script>
<script src='/common.module.js'></script>
<script src='/Page/Feed.module.js'></script>
{extends file=$render->getLayoutPath('Page/Abstract/two-column.tpl')}

{block name='content-title'}Feed{/block}
{*
  ...
*}

to this

<script src='/Page/Feed.module.js'></script>
{extends file=$render->getLayoutPath('Page/Abstract/two-column.tpl')}

{block name='content-title'}Feed{/block}
{*
  ...
*}

by moving all of the common files as common.module.js and vendor.module.js to some parent common template because these files are required by every page template.

[tomaszdurka]

Just briefly looking on this post. You are using require('relative path to the CM_Component_1'); a lot, but you probably can simply require('CM_Component_1'); once it's defined within that component.

[tomaszdurka]

When I looked into these solutions I thought about first defining (amd) packages which could be required not by using relative path, but simply view name e.g. SK_Component_Name. Not sure if we can define modules containing both js and css files in amd (guess not).

I understand webpack is about to wrap couple of js and css files together. Can we create inbetween webpacks for storing css and js files?

---

I don't think we should ever define dependencies like below:

 Feed: {
      dest: "./Page/Feed.module.js",
      dependencies: ["./Page/Abstract.js", "./Page/Feed.js", "./Page/Feed/default.css", ...]
    },

This bring so much extra effort whenver simple change to page/component will be added.

[njam]

Copying Aleksei and Tomasz' notes about Webpack:

Webpack doesn't fit well into CM framework

• webpack compiles whole client-side code at once - it's not possible to execute server-generated code
• define static endpoints for each view
• during loadPage we need two requests (first for getting page class name, second for loading dependencies)
• we need to define smarty-code dependencies in javascript files (or find other hack)