[bz5667423] Add support for including files in a configuration bundle

[jlecomte]

Use cases:

• Configuration bundles can get very big. The ability to split a large configuration bundle into multiple files makes it easier to wrap your head around and maintain your application configuration.
• Configuration bundles can contain a large section common to many applications, and a small section specific to a single application. The ability to split a large configuration bundle into multiple files makes it possible to share the common portion easily.

More specific use case applied to my project:

My team maintains a large number of independent apps (think 100+). Each app is a separate mojito application. Each app has some application level configuration which is a ycb bundle in a file named application.json. 90% of the content of that file is shared among all our apps. Therefore, it is not only stupid but dangerous to have 100+ copies of that file, each with their own sets of changes to support app-specific customizations.

Note: Yahoo!'s internal YCB library already supports "import_dimensions" to include dimensions files and "import_settings" to include settings files. See internal ticket #5667423

[jlecomte]

Additionally, it would be nice to extract the YCB library and make it its own independent npm package which mojito can depend on.

[drewfish]

This issue should probably move to https://github.com/yahoo/ycb/issues.

[mridgway]

ycb is currently not doing any file reading logic; data is inserted into the library by passing in a full javascript object. Do we need to move all filing reading logic into ycb in order to make this work? This would be an API shift from what I can tell.

[jlecomte]

I'm not sure what the final solution should be. Please, tell me if the use cases are clear enough though. I'd be more than happy to sit down and walk you through them.

[drewfish]

The current approach is nice because it allows the caller of YCB to deal with filesystem issues (sync vs. async, etc). However, it doesn't allow for the "include" feature that we'd like to add.

If we change YCB to read the files itself, the interface will have to be asychronous of course (perhaps optionally sync as well).

Using the object-oriented interface, we could perhaps get both approaches:

var ycbfile = new YCB({...options...});

ycbfile.loadJSON({...json...});
/// --- or ---
ycbfile.loadFile(path, callback);
/// --- or ---
ycbfile.loadFileSync(path);

/// --- then later
var config = ycbfile.read({...context...});

[caridy]

will a custom specs definition per mojit help in this case to get better organization/granularity?

[drewfish]

Even with specs-defined-in-the-mojit (which is possible today in fact), I think there might be a good usecase for splitting the application.json into parts.

[caridy]

PR #905 adds support for multi part application.json by using applicationConfigFiles in the master section of application.json to specify relative files that should be parsed. import_settings is still pending though.

[caridy]

this was solved a while ago.