how to: shared dependencies, separate pages

[stephenmathieson]

i've been using component-bundle for a chrome extension. these bundles (pages?) are not shared and have nothing to do with one another. i'm trying to port my build to use component 1.0.0, but am having an extremely difficult time.

tldr;

i've got 4 "pages":

• background (has remote deps)
• content (has local and remote deps)
• options (has remote deps)
• browser-action (has no deps)

each of my pages are effectively "boot" -- they all do their own bootstrapping, etc.

two of my pages (background, content) share remote deps, but cannot be included together. with the current API, is there any way to accomplish this?

[timaschew]

that is confusing:

these bundles (pages?) are not shared and have nothing to do with one another

two of my pages (background, content) share remote deps

if you don't have shared dependencies, you still need to use the boot component, at least for the require implementation, then you should be able to include your background and content together.

if you have shared dependencies, they will be added to boot.

if you can't use background and content together because of your own bootstrapping, then you need to decouple them.

[stephenmathieson]

perhaps my verbiage isn't quite clear: the two pages both depend on someuser/somecomponent@x.y.z. outside of that, nothing is shared.

it's not my bootstrapping; it's the way chrome extensions must be separated. see the high-level docs for a more thorough explanation.

regardless, to combat the weird "boot" requirement provided here, i've removed the usage of bundler.js entirely:

var Batch = require('batch');
var c8 = require('./component.json');
var builder = require('component-builder');
var resolve = require('component-resolver');
var bundles = c8.locals; // background, content, etc.

resolve(__dirname, { install: true }, function (err, tree) {
  if (err) throw err;
  var pending = bundles.length;
  bundles.forEach(function (name) {
    var bundle = tree.locals[name];
    var batch = new Batch;
    [
      buildScripts,
      buildStyles,
      buildFiles
    ].forEach(function (fn) {
      batch.push(function (done) {
        fn(bundle, done);
      });
    });
    batch.end(function (err) {
      if (err) throw err;
      if (!--pending) {
        /* ... */
      }
    });
  });
});

function buildStyles() { /* ... */ }
function buildScripts() { /* ... */ }
function buildFiles() { /* ... */ }

i still feel that a less opinionated bundling option should be provided. i'd be happy to contribute one if you're interested.

[timaschew]

does the bundler work for you? Because can't get it working, see here

[stephenmathieson]

Yeah it was working; just not doing what I wanted it to.

[timaschew]

okay, can you please show me how your directory strucutre looks like and the component.json files. Or check how this structure differs from yours?

[stephenmathieson]

my structure looks like this:

├── History.md
├── Makefile
├── Readme.md
├── build.js
├── component.json
├── lib
│   ├── background
│   │   ├── component.json
│   │   └── index.js
│   ├── browser-action
│   │   ├── 19.png
│   │   └── component.json
│   ├── content
│   │   ├── component.json
│   │   ├── index.js
│   │   └── styles.styl
│   ├── some-local-component-1
│   │   ├── component.json
│   │   ├── index.js
│   │   └── styles.styl
│   ├── some-local-component-2
│   │   ├── component.json
│   │   ├── index.js
│   │   └── styles.styl
│   ├── some-local-component-3
│   │   ├── component.json
│   │   ├── index.js
│   │   └── styles.styl
│   ├── some-local-component-4
│   │   ├── component.json
│   │   ├── index.js
│   │   └── styles.styl
│   └── options
│       ├── component.json
│       ├── index.js
│       └── options.styl
│       └── options.jade
└── manifest.json

and my component.json:

{
  "name": "extension",
  "paths": ["lib"],
  "locals": [
    "options",
    "content",
    "background",
    "browser-action"
  ]
}

[timaschew]

thanks I have the same structure, how your build.js looks like? and can you show me your dependencies (package.json) with the version of bundler, resolver and builder? I'm using:

    "component-bundler": "0.0.3",
    "component-resolver": "~1.1.6",
    "component-builder": "~1.1.6"

[stephenmathieson]

the build.js is the script above. as i couldn't get the bundler to work as i expected it to (hence this issue :p), i am no longer using it. the version would have been latest as of wednesday, which i think is 0.0.2

[stephenmathieson]

also, i couldn't get this to work, so was using builder.styles(bundles[bundle]).use('styles', builder.plugins.css()) and builder.scripts(bundles[bundle]).use('scripts', builder.plugins.js()).

[timaschew]

Now my boot.js have the entire code and dependencies, but the require implementation is missing. AND the other bundles (pages) still contain only undefined.

Is your problem related to my issue?

[stephenmathieson]

nope, i never had an undefined :/

[timaschew]

@jonathanong any feedback? how to use bundler? @stephenmathieson confirms, that it doesn't work with this line from the readme: build.scripts(bundles[name]).build(function (err, content) {})

and for me the other syntax:

 build.styles(bundles[name])
    .use("scripts", build.plugins.js())
    .end(function(err, content) {})

doesn't generate the desired output

[clintwood]

@timaschew, @stephenmathieson not sure if this will help... I've pushed a stripped down project using component-bundler here clintwood/koa-component-bundler-example. It works form me as advertised :)

[jonathanong]

@stephenmathieson you need to use the plugins. you're not using any plugins in that example.

[jonathanong]

btw this package should be more instructional than actually used. create your own bundlers from the examples! might be easier than trying to use these haha

[stephenmathieson]

Yup I spent ~2h trying to get this thing to work, gave up, and wrote my own in a few minutes. This said, should the readme suggest that this thing doesn't actually work and users should roll their own impls?

[stephenmathieson]

... Or should working bundlers be put together?

[jonathanong]

feel free to add more. i'll make it more explicit that this is more for education

[timaschew]

@jonathanong

@stephenmathieson you need to use the plugins. you're not using any plugins in that example.

he did, I didn't do it in my example, or you don't mean this .use("scripts", build.plugins.js()) ?

@clintwood thanks! that worked, at the first glance I don't see any differences in the builder script, except that you're using generators, I'll analyze it later.

@stephenmathieson @jonathanong The readme should show an minimal example that works. Currently it doesn't because the plugins are not used.

[timaschew]

@clintwood I tried to refactor, exclude koa, but now it doesn't work anymore: get this error now:

undefined:1
undefined
^
SyntaxError: Unexpected token u
    at Object.parse (native)
    at appBundler (/Users/awilhelm/dev/koa-component-bundler/lib/app-builder2.js:45:21)

line 45 is this:

var json = JSON.parse((yield cofs.readFile(comp)));

I add a build.js and app-builder2.js file (remove koa and HTTP stuff) https://github.com/timaschew/koa-component-bundler/commits/master

here is the build.js

var path = require('path');
var bundler = require('./lib/app-builder2');

var g = bundler({
  root: path.join(__dirname, 'private', 'app'),
  build: path.join(__dirname, 'public')
});

var gen = g();
while (!(info = gen.next()).done) {
  console.log("yield from gen()");
}

and here the diff for the bundler usage

@@ -35,9 +35,9 @@
   // koa middelware
   return function* appBundler(next) {
     // filter on HTTP GET/HEAD & *.js|css resource targeting build dir
-    if (('GET' !== this.method && 'HEAD' !== this.method) || !/\.(js|css)$/.test(this.path) ||
-      path.relative(options.build, this.path).slice(0, 2) != '..')
-      return yield* next;
+    // if (('GET' !== this.method && 'HEAD' !== this.method) || !/\.(js|css)$/.test(this.path) ||
+    //   path.relative(options.build, this.path).slice(0, 2) != '..')
+    //   return yield* next;

@@ -116,6 +116,7 @@

-    yield* next;
+    // yield* next;
+    yield 'finished';
   }
 }

[timaschew]

this issue #7 becomes mixed with my issues, which are maybe not equal with yours @stephenmathieson is your issue solved now?

[stephenmathieson]

no, not really. this was more of a question/feature request than an a "something is broken" issue.

i've got a working builder now, so i'm not too worried about it anymore. i'll close (though unresolved) if you guys want..

[clintwood]

@timaschew, I've lost track of what your real issues are... anyhow, I've refactored the sample I put up for you (in the express branch) to use Express over here. So it doesn't use generators any more... sad really! :)

[timaschew]

@stephenmathieson I improved the readme; I think this is your case, right?

There are cases, where you don't have a boot component (none or multiple). In other words: if your first component has scripts that shouldn't be mixed with other bundles. You can define your locals of your main component: ["common", "bundle-a", "bundle-b", "bundle-c"] and create a file with an empty object ({}) at this path lib/common/comonent.json.

from my fork (PR is out): https://github.com/timaschew/bundler.js#how-it-works

[timaschew]

@clintwood thanks! I thing this is the problem: https://github.com/component/builder2.js/issues/62 I think koa and express catching this error or undefined value

[bodokaiser]

I have a question regarding nested bundles with shared dependencies as in e.g.:

/common
    /component.json
    /boot
        /index.js
        /component.json
    /navbar
        /index.js
        /style.css
        /component.json
/projects
    /component.json
    /create
        /component.json
        /form
            /index.js
            /style.css
            /component.json
    /model
        /component.json
        /index.js

We have two main bundles common (for shared logic, layouts, navbar, whatever) and projects. projects has again sub bundles which can be shared between.

This would generally fit with the setup in bundler.users. However I read that all (third-party) dependencies (e.g. component/domify) would now be compiled in both main bundles. So as I need to include a common build on every site anyway I would have doubled dependencies for everything what common and projects use both.

Is this correct or is the build script so "intelligent" and just moves all dependencies to common?

[bodokaiser]

Wow amazing. I just tried the above setup (even a bit more nesting actually) and it just worked. It just worked. I can't believe it :)

So infinite nesting should work with the bundle out of the box.

However the only question mark which I am still left with is what the purpose of bundles.userPages is. I mean the above setup is more or less the same as drafted in /lib/user_pages.js but I only get it to work with bundler.pages.

If I use bundler.userPages there is no common bundle anymore and all dependencies are missing. But anyway what is the point of bundler.userPages?

[timaschew]

@bodokaiser So you have two main components and need to run the build process for each of them, right? So your main components are completely independent. So some dependencies might be defined in both of your main components (but this is what you want, right?) If the dependencies of your common and projects component are disjoint then at least the require implementation exist in both.

I think you setup is kind of special. I would prefer to run the build process only one time and would like to have shared dependencies and I think this should work with bundler.userPages.

If you app isn't a single page app, you usually are fine with bundler.pages but if the generated files are too big or you want to split them because of other reasons - for instance if you want to distinguish between logged in and anonymous users - then you need bundler.userPages

@jonathanong do you agree with this explanation?

[bodokaiser]

@stephenmathieson not quite right :(

Dependencies are not disjoint. They are kept in common.js. Projects.js just extends them for the projects page (so there also can be a settings.js for a settings page and so on)".

So I still do not get the difference to userPages

[stephenmathieson]

Not quite right? Huh?

User pages make little sense and documentation for this tool is lacking. Basically, @bodokaiser, you should just roll your own bundler script. It's pretty simple; I've got an example above.

[stephenmathieson]

Closing; this is issue is unfocused and my original problem has been solved.

[bodokaiser]

@stephenmathieson do you have a link to how your example looks now? The one mentioned on this issue was commented with not working?

[stephenmathieson]

@bodokaiser I'll add my script to the module in the morning. Sorry for the delay!

EDIT: it works fine; this thread is just really hard to follow.

[stephenmathieson]

@bodokaiser see this gist. it's a trimmed down version of our build (no jade, stylus, etc. included).

[bodokaiser]

@stephenmathieson ah you are still working with callbacks xD

My looks similar but still requires component-bundler. I am also not sure what the purpose of batch is (in general).

https://gist.github.com/bodokaiser/ecf3b7a72d6d1929d96f

[stephenmathieson]

Haha, yep. None of our production stuff (including builds) runs on unstable node :p

Batch makes running stuff with callbacks in parallel manageable.

On Jun 23, 2014, at 10:08 AM, Bodo Kaiser notifications@github.com wrote:

@stephenmathieson ah you are still working with callbacks xD

My looks similar but still requires component-bundler. I am also not sure what the purpose of batch is (in general).

https://gist.github.com/bodokaiser/ecf3b7a72d6d1929d96f

— Reply to this email directly or view it on GitHub.