search

chiasm-project / chiasm

A browser based environment for interactive data visualizations.
MIT License
184 stars 27 forks source link

Migrate Chiasm to JSPM #29

Closed curran closed 8 years ago

curran commented 9 years ago

Chiasm plugins should each live in their own separate git repositories, and be included in a project via JSPM. This is a similar approach to the Gulp system and plugin architecture.

Hypercubed commented 9 years ago

Are you planning to switch completely to JSPM/SystemJS? Do you/we need to support requirejs in chiasm and chiasm plugins?

curran commented 9 years ago

Yes, I want to eliminate RequireJS completely from the Chiasm project, and use JSPM and SystemJS instead. The main reasons are:

The one thing I'm not exactly sure about is how to bundle a Chiasm application for inclusion in a production flow, like the Rails Asset Pipeline (which Chiasm must support). It should be possible to use JSPM to create a self-executing bundle that exposes Chiasm as a Browser global AND supports dynamic instantiation of plugins included in the bundle. I haven't seen this work yet, but because JSPM creates bundles that include a version of System.import that can load bundled modules, I'm convinced that it should be feasible.

I would like to go down this road, so if you have the energy to try swapping out RequireJS for JSPM and SystemJS completely, then go for it!

Also, you might want to check out the tooling that the new version of D3 is using - see d3-selection and d3-bundler. Based on this work, it feels like Rollup would be a great bundling solution, but it's not clear to me if it's possible to use JSPM with Rollup, or if that would give a significant advantage over the JSPM/SystemJS bundling strategy.

Here are a few related threads:

curran commented 9 years ago

On second thought, I might just want to go with CommonJS and NPM for publishing Chiasm-related modules, then use Browserify for bundling. It seems these tools have a lot more traction than JSPM, they are more mature and pleasant to work with, and they achieve almost the same thing as JSPM (single install, single require).

After having tried out JSPM a bit, I find the following two things to be the most questionable aspects:

Even for ES6, it seems that many people are using a workflow where they transpile to ES5 first, then publish to NPM (see this thread Reddit: What's the best way to publish ES6 modules?.

curran commented 9 years ago

In evaluating all of the options for bundling and plugin authoring for Chiasm, I ended up created this screencast on JS modules and build tools.

I'm leaning more and more toward NPM + Browserify for splitting up the Chiasm codebase across repositories, mainly because this is a relatively mature toolchain that works extremely well. This means Chiasm will impose NPM as a plugin publishing and install mechanism, and force developers to deliver plugins as NPM modules (CommonJS). This does not preclude authoring plugins as ES6, because the ES6 code can be transpiled to ES5 before publishing to NPM, which is already a fairly common pattern.

In a way, this is less tooling to impose as compared to JSPM, because to use JSPM, one already needs to be using NPM, and additionally JSPM introduces the "config.js" file, and dependencies on ES6-related things. I'm just not convinced JSPM is stable/mature enough for use as the Chiasm bundler, and the space for authoring modules in ES6 and transpiling to ES5 is vast.

I'm planning to start breaking out Chiasm plugins soon, using NPM.

Hypercubed commented 9 years ago

This is a reasonable approach. NPM/CJS modules are easy to install using JSPM as well, if the developer chooses. I would hope, at least, that the plugin system continues to support SystemJS.

Although I don't know if NPM + Browserfy + RequireJS is any less tooling that NPM + JSPM + SystemJS.

curran commented 9 years ago

Thank you for your input. This new approach would eliminate RequireJS, so the tools would only include NPM + Browserfy.

I would however be happy to keep support for dynamic loading with SystemJS if that is something you'd like to use.

My main concern is proper bundling for production, and splitting out the codebase into modules in such a way that bundling is straightforward. The outline for the bundling strategy is here in this README - https://github.com/curran/chiasm-bundle

On Wed, Jun 17, 2015 at 7:07 PM, Jayson Harshbarger < notifications@github.com> wrote:

This is a reasonable approach. NPM/CJS modules are easy to install using JSPM as well, if the developer chooses. I would hope, at least, that the plugin system continues to support SystemJS.

Although I don't know if NPM + Browserfy + RequireJS is any less tooling that NPM + JSPM + SystemJS.

— Reply to this email directly or view it on GitHub https://github.com/curran/chiasm/issues/29#issuecomment-113010836.

curran commented 9 years ago

The more I consider various use cases, I think that loading plugins on-demand is actually a fundamental requirement for Chiasm. In some cases it makes sense to bundle the required plugins, but for more general use cases, it may not be possible to know ahead of time which plugins will be required. If the set of plugins grows to hundreds some day, it would not make sense to bundle all of them with all of their dependencies if only a few are ever used.

I think I'm going again down the road of evaluating JSPM, and trying to make it work for Chiasm both with bundling and with dynamic module loading. I was turned off initially by the huge bundle produced by JSPM, and how it just felt a little bloated all around, but I think since that time there has been tons of work on JSPM that address some of the issues I had.

Hypercubed commented 9 years ago

I think the benefit of JSPM bundle vs other tools is that:

1) unlike gulp-wiredep (for example) it only includes modules that are actually used by analyzing the module tree. So at the app level you can JSPM install as many modules as you like and only ones that are used are bundled. Of course, as we found, this is difficult for Chiasm plugin loading. Shomewhere you need to specify what plugins are used (https://github.com/Hypercubed/Project-Chi/blob/master/gulpfile.js#L12). 2) Other tools, from what I can see, just do a textual include. jspm bundle keeps modules as modules. This means some added noise in the jspm bundle. However, I agree at the plugin level rollup or Esperanto should probably be used.

Two things to note about the jspm bundle. You can turn on minify and make sure you source maps are not inline.

curran commented 8 years ago

Here's a new idea: I think it would be cool to make a SystemJS module loader plugin for Chiasm. When the plugin is instantiated in a Chiasm configuration, it can be set up with local (possibly in jspm_packages) or remote (CDN) paths for other Chiasm components. For this, there will need to be a hook in the Chiasm API that would allow plugins to register and unregister themselves as module loaders. This would also make it possible to, for example, build another totally separate loader plugin that uses RequireJS.

Hypercubed commented 8 years ago

Honestly I haven't looked at the project in detail lately so I am falling behind. I'm not sure I follow how a SystemJS plugin can help. If plugins are all JavaScript files then one can load all plugins explicitly:

      var chiasm = Chiasm();

      chiasm.plugins.layout = require('ChiasmLayout');
      chiasm.plugins.links = require('ChiasmLinks');
      chiasm.plugins.dataLoader = require('DataLoader');
      chiasm.plugins.pieChart = require('PieChart');

JSPM builder would love this. Each plugin can load and register it's dependencies the same way, possibly under an alias to avoid collision.

Hypercubed commented 8 years ago

@curran Reading this thread and looking back at our previous discussions I can't recall why lazy loading of plugins was needed. I added the SystemJS loading support to the Chiasm code and went through some trouble in project-χ (here: https://github.com/Hypercubed/Project-Chi/blob/dc759095ad1cff7cd4ad850857389f084d2bcedd/gulpfile.js#L12) to get the chiasm plugins bundled so that they can be loaded by the loadPlugin method (https://github.com/chiasm-project/chiasm/blob/master/src/index.js#L166).

But why?

The approach you are using in your examples (simply add the plugin constructors to the chiasm instance plugins hash) seams much more maintainable and robust. A developer that has installed chiasm and plugins (using jspm or whatever) needs to create a chiasm instance in the application. For each instance the necessary plugins should be added. Am I missing something?

Hypercubed commented 8 years ago

I've updated project-χ with the latest chiasm version (including new layout, links, and bar chart from your example): https://github.com/Hypercubed/Project-Chi/tree/chiasm/app/components/examples/chiasm

Everything installed fine using jspm specifying the npm registry (jspm install chiasm=npm:chiasm). Installing directly from github would require adding the registry to package.json. I'm explicitly importing the plugins (chiasm-layout and chiasm-links) and adding to my chiasm instance. So loadPlugin is not needed at all. I had to jspm install chiasm-component also because the barChart plugin is local and requires it. I had to add lodash to window because we are missing the import in chiasm-layout (see https://github.com/chiasm-project/chiasm-layout/issues/1).

curran commented 8 years ago

Right, I think this has to do with the history of the project. Originally, Chiasm was using Require.js to dynamically load plugins. Then the need for bundling came up, and I had to re-think how the plugins were being loaded. At this point the chiasm.plugins object was added, so bundling tools can know which plugins to bundle. Currently, using chiasm.plugins is the way I'm making new examples, and this works well. Indeed, I think it is more maintainable and robust than the previous solutions that depended on either Require.js or System.js.

However, in a way it would be nice to be able to include a plugin in the configuration only and have everything "just work". With the new examples, each plugin needs to be defined in three places:

  1. loaded with a Githubissues.
  2. Githubissues is a development platform for aggregating issues.