Plugin Proposal

[ilinkuo]

Current state

F2 doesn't really have any real guidance on how to create a plugin. The only method to aid in that is F2.extend, which creates a new subnamespace within F2. The extend method is inadequate if one wants to add to F2.Events or F2.Constants because the extend method will simply override the entire namespace. While one could just directly manipulate F2, that is inelegant and prone to errors.

Base Proposal

Definition

An F2 plugin is a plain object or a function returning a plain object. The function may take a single argument which is the current F2 before extension. The current F2 argument allows the plugin to apply aspects to existing F2 methods.

How plugins work

Plugins will be installed into F2 via F2.install(pluginsArray) where each element of the array is either an object or a function as specified above. The install() function will return the modified copy of F2, leaving the original F2 intact. The install() is optional may be run multiple times before init() is called, but cannot be run after init() The plugins in the array will be copied in the order they are listed, so in the event of collisions, the latest plugin wins. Collisions are minimized by copying the properties at the level of the leaf using jquery.extend() with the deep option set to true. The use of jquery.extend() allows a plugin to easily add its code as well as its custom events and constants to F2.

For a proof of concept see this jsfiddle

Considerations

The implementation should be done in a way that will be easily compatible to a move to AMD.

Extension 1

The example fiddle above doesn't check for compatibility between versions. It also doesn't register the plugin with F2 in a way that a particular F2 instance can be queried about the plugins & versions it has installed.

Extension 2

Since F2 is currently built up via mixins, the plugin approach is compatible. Consider restructuring F2 as core + standard set of plugins.

[rwadkins]

So, if I had multiple plugins, I would call install on the F2 instance returned by a previous F2.

[ilinkuo]

@rwadkins Yes. This potentially allows multiple F2 instances with different plugins to coexist at the same time. As F2 is now a singleton, there isn't any problem. But if multiple instances are in its future, some of the method implementations will have to be rewritten to account for this -- essentially, internal references to F2 should be eliminated in favor of this references. This also potentially allows a Container and an App to receive different but connected instances of F2.

[markhealey]

Hi @ilinkuo,

In looking at your initial comment:

The extend method is inadequate if one wants to add to F2.Events or F2.Constants because the extend method will simply override the entire namespace. While one could just directly manipulate F2, that is inelegant and prone to errors.

It sounds like you're not familiar with the inner workings of the F2.extend method. I have made a small jsFiddle to demonstrate how using F2.extend does not overwrite the existing namespace. The extend method does exactly what it says with the option to overwrite existing properties. Check out this relevant code in /src/F2.js.

Is this unexpected or insufficient for your needs? Additionally, it sounds like there isn't enough documentation to support developers extending F2.

F2 doesn't really have any real guidance on how to create a plugin.

Could you elaborate on that or make suggestions on what would be helpful to add to the docs?

Thanks.

[ilinkuo]

Yes, @markhealey . You're correct, I amended the jsfiddle example to do what I wanted it to do, which was to update within a namespace. I read "namespace" and understood that as a root namespace, but the method actually allows namespace values such as "Constants.Events" and will drill down appropriately. The docs can be amended to indicate that "namespace segments" can be used.

There are things missing from just providing an F2.extend() that I would like to see in the framework, although, to be honest, F2's plugin philosophy seems to be resemble jQuery's plugin with no restrictions whatsoever, and that seems to have worked well for jQuery. So, if you're following jQuery's lead, then maybe you don't need any documentation.

The two things I would like to see which my original proposal addresses:

• Only allow the Container, not the App to install plugins. So extend() can only be called before F2.init().
• Check version compatibility when installing plugin.

[markhealey]

Good update to the jsFiddle, maybe that becomes the documentation! Yes, our intention was to leave it wide open similar to how jQuery works.

Two questions:

1. Allowing only the Container Developer to write plugins seems like an arbitrary restriction. What problems do you foresee in letting both sides implement plugins?
2. I like this idea at a high level but what would happen if the version didn't match? Simply not load the plugin? And if any apps are heavily dependent on that plugin, for whatever reason, then they'd not load either?

[ilinkuo]

Our current dock allows up to 20 modules. If the main content area were also modularized similarly, that would be another 5 or 6. While only a few modules are F2 Apps, it would seem that future fully F2 Containers could have 25 F2 modules. In the worst case, all 25 modules are different modules from different providers. If Apps are allowed to freely install plugins in the global F2 framework, I would expect the incidence of bugs due to monkey-patching to be quite high. The burden of initially diagnosing and delegating these bugs would also fall on the Container. These monkey-patching bugs also have a high cost of diagnosis because they're difficult to reproduce as they often depend on Apps' nondeterministic load order.

The reason for restricting Apps and not Containers is because the debugging burden is assymetric and increases nonlinearly with the number of Apps allowed in the Container's system. That's my answer to your first question

To decrease the burden, do as much by automated verification as possible. Version checking is one easy check to automate. Dependency checking is another easy check (once conversion to AMD is done). With AMD, if the App's dependencies aren't available, then the App doesn't load. From the Container's point of view, not allowing the App to load is clearly preferable to allowing the App to load and then having errors.

With the current F2 loading system, I prefer to have the only Container do the loading. With an AMD-based loading system, there's a lot more leeway to distribute loading responsibility, as AMD can allow you to loading different versions of jQuery or other properly written AMD libraries at the same time.

[montlebalm]

The install() function will return the modified copy of F2, leaving the original F2 intact.

I can't say I see the practicality in having multiple instances of F2 on the page at once. With different instances you wouldn't be able to broadcast an event to every app since EventEmitter maintains its own internal state.

If Apps are allowed to freely install plugins in the global F2 framework, I would expect the incidence of bugs due to monkey-patching to be quite high.

I agree that the chance of overlapping plugin namespaces from different providers could be quite high. Is it wise to have plugins at all? The only purpose seems to be to coalesce functionality around a single global. If F2 is designed to encourage modular architecture, perhaps plugins are not a worthwhile feature.

In version 2, we're kicking around the idea of allowing the container to specify how modals are rendered. In that case, you might wish to have a F2.BootstrapUI plugin. Instead, why couldn't we simply do:

// Proposed "config" method to replace "init"
F2.config({
  ui: BootstrapUI
});

We can achieve the same code sharing without having to modify the F2 instance. I understand the convenience of plugins for a single developer on a site they control, but that's not the situation we're facing.

[ilinkuo]

I can't say I see the practicality in having multiple instances of F2 on the page at once. With different instances you wouldn't be able to broadcast an event to every app since EventEmitter maintains its own internal state.

That's actually precisely the point of multiple instances. It's the first step to enabling distinct non-interfering event channels. The current F2 eventing system is too insecure -- any app can listen in on any other app's events. Individual F2 instances serve as sandboxes to allow each app to isolate its events and its plugins. A further specification would allow each app to open up its sandbox/channel selectively.

The only purpose seems to be to coalesce functionality around a single global. If F2 is designed to encourage modular architecture, perhaps plugins are not a worthwhile feature.

Plugins done in the jQuery way have this drawback. A different plugin implementation could be designed without this problem.

In version 2, we're kicking around the idea of allowing the container to specify how modals are rendered.

I may be missing your point, but your example seems to only switch out the implementation of ui. That doesn't address the idea of plugins which is to extend the functionality of F2. How does your example work if I wanted to add an entitlements plugin functionality to F2?

We can achieve the same code sharing without having to modify the F2 instance.

The reluctance to modify the F2 instance, I think, stems from the fact that it is a singleton, and singletons are regarded as sacred. I actually dislike the fact that the F2 instance is a singleton, and I would like to move towards the multi-container vision as explained in this blog post of mine. http://ilinkuo.wordpress.com/2013/02/11/the-borg-container-assumptions/

Singletons are very convenient, but it's beginning to be recognized that they are an anti-pattern. Even something such as jQuery recognizes that it may be one of several jQuery instances on the page and reacts accordingly. Also, one of the major advantages of AMD is that it has configuration mechanisms to allow multiple versions on the same page (see http://requirejs.org/docs/api.html#multiversion). Thus, one could conceivably have multiple modules on the same page, each written to a different version of F2. This could happen only if F2 re-architects itself to remove the singelton assumptions. I wrote the above plugin API proposal with that in mind.

but that's not the situation we're facing

The situation that caused me to write the above proposal was one where the F2 App developer added a jQuery plugin which was incompatible with the rest of the code on the page. The jQuery plugin in question needed only a repackaging to be compatible with F2 and our code. The best way to do this, in my opinion, would be to repackage that code within an F2 plugin to ensure compatibility. However, in trying to write a plugin, I found a lack of guidance. A good plugin architecture seemed to me the best way of extending F2 functionality nonintrusively while having the highest chance of forward compatibility.

[montlebalm]

Individual F2 instances serve as sandboxes to allow each app to isolate its events and its plugins.

Is the suggestion that apps make their own instances of F2? How would they share it across apps from the same domain? It seems to me that the container would need to be in charge of instantiating F2 and passing it off to quarantined apps. If the app has "F2" as an AMD dependency, how does the correct version of F2 get passed to it?

JsFiddle of F2 instance "spawning"

I like the idea of multiple instances, but there are a number of problems with that approach. For example, if the container wants to reload an app it must first determine which spawned instance to load it with. This adds an extra layer of complexity for container developers.

As for plugins, it sounds like you want to be able to modify the existing implementation of F2 instead of adding to it. You could add hooks inside the code to look for additional implementations, but it sounds like you could easily open up some serious security loopholes.

It might be more illuminating if you could provide working code examples that illustrate the goals you have in mind. Specifically, a way for multiple instances to be created with selective 2-way interconnectedness. Also, a plugin system that allows for the implementation of the library to be changed without compromising security.

[ilinkuo]

A lot of excellent, excellent questions!

Is the suggestion that apps make their own instances of F2?

No. I had in mind that the Container has a policy configuration that determines which Apps get their own F2 and which Apps share, so it would be the Container which could make new F2 instances, not the App.

I also had in mind that the Container's copy of F2 be different from the App's copy of F2, and that the differences would be controlled by a policy. For example, the Container could decide which Apps are allowed to call F2.extend() and what were allowable values to call it with.

How would they share it across apps from the same domain?

The Container can keep track of the F2 instances it has created, and say that F2 App instances from the same domain get the same F2 instance. That's a policy decision. There are lots of ways to do this.

It seems to me that the container would need to be in charge of instantiating F2 and passing it off to quarantined apps. If the app has "F2" as an AMD dependency, how does the correct version of F2 get passed to it?

Well, currently F2 is not AMD yet, so it depends the path chosen to AMDify it. In the way that seems the most straightforward to me, I would take the manifest response and rewrite it as an AMD declaration. However, since there is an intermediate rewriting step, it's easy to rewrite the "F2" dependency as something like "com.markit-on-demand/F2" or "com.tdameritrade/F2".

Even if F2 were already fully AMD, the better AMD implementations out there support common config (https://github.com/amdjs/amdjs-api/wiki/Common-Config) and allow you to specify multiple "versions" of F2. There are multiple mechanisms such as packages and paths and maps to enable the loading.

I like the idea of multiple instances, but there are a number of problems with that approach. For example, if the container wants to reload an app it must first determine which spawned instance to load it with.

There are multiple architectures that could lead to multiple instances, so the answer depends. I actually favor one which involves multiple Containers on the same page as well. I don't think the problem of determination is that hard with the right architecture -- after all, there are many thousands of DNS Servers in the world and we seem to manage to find the right one without much difficulty.

This adds an extra layer of complexity for container developers.

I would think that support for multiple F2 instances would be optional. A Container need only support it if it needs some relevant feature which requires having multiple Containers.

As for plugins, it sounds like you want to be able to modify the existing implementation of F2 instead of adding to it. You could add hooks inside the code to look for additional implementations, but it sounds like you could easily open up some serious security loopholes.

Right now I'm adding hooks directly into F2 code, which is very ugly and not likely to be forward compatible. I'm going to rewrite them as plugins to ease my upgrading burden. Once that's done, I'll pick a few to contribute back to F2 as appropriate.

It might be more illuminating if you could provide working code examples that illustrate the goals you have in mind. Specifically, a way for multiple instances to be created with selective 2-way interconnectedness. Also, a plugin system that allows for the implementation of the library to be changed without compromising security.

I'm hoping to make some pull requests, hopefully within 4 month time frame, though neither of these examples are on my list.

[montlebalm]

The more I think about it the less I think multiple instances are a good idea. It's a convenient solution for solving the monkey patched plugins, but I wonder if the harm outweighs the good.

I've been thinking about your suggestion of using an AMD loader to differentiate "td/f2" and "mod/f2" across apps, but doesn't that open up the exact same security concerns from before? A nefarious app couple specify a slew of common vendor prefixes and wind up with your instance of F2.

The best thing I've come up with so far is to keep the F2 singleton and implement a sandboxing contract between apps that limits exposure. Here's a quick example:

// Put 2 apps in a sandbox together where they can share everything
F2.sandbox(["com_mod_luke", "com_mod_leia"], {
  // desired privacy options
});

F2.sandbox(["com_mod_leia", "com_mod_han"]);

The results of the sandboxing are:

• "luke" can share events with "leia"
• "han" can share events with "leia"
• "leia" can share events with "luke" and "han"

We could try and keep the sandbox method locked down to the container, so apps wouldn't be able to sign themselves up for someone else's events. It also gives us the ability to fine tune the customization if necessary. We can do whitelisting, blacklisting, regex appId matching, or whatever we feel is the best solution.

This wouldn't require too many modifications to the source. We would just add some checks inside the event system to look at who the sender and receivers are and apply the rules accordingly.

The only problem I see is that it requires the container to implement security. App developers still can't be 100% it's being done correctly and couldn't feel totally secure.

Ignoring the plugin issue, do you think this is a viable solution to the "event spying" question?

[ilinkuo]

The more I think about it the less I think multiple instances are a good idea. It's a convenient solution for solving the monkey patched plugins, but I wonder if the harm outweighs the good.

I will try to convince you that many F2 instances are a necessary thing if you care about security, by first trying to convince you that 2 F2 instances are a good idea -- one for the Container and one for the App. I think you are alluding to that later on when you say

We could try and keep the sandbox method locked down to the container

How would you restrict that sandbox method without having separate instances for Container and App? Having two instances would also solve the problem that there are a few more methods than sandbox that need to be restricted from the App.

However, If you think about it, if the App and Container share the same F2 instance, then the App is able to aspect around the F2.Events.emit() method to snoop each and every event, and there is no sandboxing technique that works against that. If you care about event security, then it is unavoidable that you give each App and Container its own instance of F2, or if not that, at least its own instance of EventEmitter.

[montlebalm]

If we go down the multiple instances track, we could try something like this:

// F2 library definition
// ----------------------------------------------------------------------------
define('F2', [], function() {
  var F2 = function(options) {
    if (options) {
      this.addPlugins(options.plugins);
    }
  };

  F2.prototype = {
    spawn: function(options) {
      return new F2(options);
    },
    addPlugins: function(plugins) {
      for (var i = 0; i < plugins.length; i++) {
        new plugins[i](this);
      }
    }
  };

  return new F2();
});

// F2 plugin definition
// ----------------------------------------------------------------------------
define('F2BootstrapPlugin', [], function() {
  function makeModal(options) {
    // return implementation
  };

  return function(F2) {
    F2.config({
      ui: {
        modal: makeModal
      }
    });
  };
});

// AppClass
// ----------------------------------------------------------------------------
define('com_green_appid', ['F2', 'F2BootstrapPlugin'], function(F2, F2BootstrapPlugin) {
  F2 = F2.spawn({
    plugins: [F2BootstrapPlugin]
  });
});

The spawn method gives you a new instance of F2 and accepts an array of functions. The plugins will be given an opportunity to add their logic to F2 after it's instantiated. This should guard against the monkey patching problem.

This diverges from the jQuery solution to plugins, which is to have the plugin automatically apply itself to the global object. Here, the developer is required to pass in the plugin to F2 directly.

This does not change how plugins are defined today. It only demonstrates a way to use them that prevents cross contamination with other apps.

[brianbaker]

I'm a little late to this party but do we have any examples of plugins? Or code that we have today that we feel should be a plugin? Are the existing examples tacking things onto the F2 namespace or are they changing certain internal parts of F2?

[ilinkuo]

Chris @montlebalm, I'm glad you're starting to come around to multiple instances. Your example shows how the App can customize its own copy of F2 without interfering with other Apps' F2 copies, so it's a good start. What concerns me is that this example code seems to head towards a world in which the Container is completely absent, and that seems wrong.

Could you please explain the role of the Container in this new plugin model?

[ilinkuo]

@brianbaker

I'm planning to rewrite all of TDA's customizations as plugins. We'll examine which are of general interest and aim to contribute those back.

Some of our plugins will add new event types to F2.Events, and add other constants. Other plugins rewrite the internal loading and view handling The internal loading modifications would be made easier by the insertion of certain hooks in the existing loading process. I plan to propose the insertion of optional hooks as a pull request

The only plugins I know of are in https://github.com/OpenF2/F2Plugins and Chris's proposed requirejs plugin.

However, in my reading of the code, F2 itself is structured entirely as plugins. All the files in https://github.com/OpenF2/F2/tree/master/sdk/src -- app_handlers.js, constants.js, classes.js, events.js, rpc.js, ui.js -- are plugins. Even the container.js itself, really.

To your point of whether the plugins extend by adding to the namespace or whether they modify F2 internally. the answer is both. It would be best if the plugin could modify F2 internals non-intrusively by adding implementations of hooks, however.

Having a proper plugin standard can ease the friction for external contributions in a safe way because changes in F2 can happen independent of the official release cycle. Internal (MOD) F2 contributors can issue pull requests directly against the current F2 releaase and the approval process seems to be frictionless. However, for external contributors, there's a whole lot of friction -- if I'm proposing making a change to modify the existing loading process, there needs to be a long evaluation. However, if I'm merely proposing adding certain backwards-compatible hooks into the loading process so it can call out to my code, then the approval is streamlined because there is no impact on existing applications. I can then add the hook implementation via a plugin and then publish the F2 plugin. F2 can evaluate the plugin, and if F2 finds that plugin useful and want to integrate into the core, it can.

[zackferraro]

@brianbaker, I have some plugins that add to the namespace and some that modify existing functionality.

Could you please explain the role of the Container in this new plugin model?

Perhaps I'm misunderstanding your question, but the container still does the XHR requests for apps and binds a DOM element as the app root. I can see the use cases of having multiple instances of F2 on the page (App Developer A doesn't trust App Developer B), but it is still useful to have a container's instance of F2 in the cases where you do want to communicate with other people's apps.

Even if it is good practice for each app to supply its own plugin dependency definitions (and not rely on the container F2 instance to already have the plugin), the container developer might have use for plugins of their own. I know I have plugins for the container that aren't used in my app code.

[ilinkuo]

@zackferraro

So let me rephrase the question as -- "what is the Container's role in the plugin workflow?"

The examples you gave of xhr and dom binding aren't part of the plugin workflow. In my original proposal, the Container serves as a gatekeeper -- ensuring that only approved and compatible plugins are loaded. Chris's proposal to me seems like a wild west of plugins -- each App is has complete freedom to implement its choices of plugin without worrying about compatibility. Providing each App its own instance of F2 does remove monkey-patching as a source of incompatibility, but there are other ways plugins can be incompatible without namespace collision.

Previous comments in this thread (https://github.com/OpenF2/F2/issues/121#issuecomment-25568405) indicates that the F2 gatekeepers seem to be skeptical of the idea the Container should be a gatekeeper of plugins, but I would argue that is because in most of MOD's projects, MOD controls both the Container and App. I don't think MOD is ever in the situation where MOD controls the Container but someone else controls the App -- this is the situation I'm in.

[zackferraro]

@ilinkuo I think it is important to allow the ability for apps to install plugins. If F2 implements your suggestion of an instance of F2 per app, then apps can be as destructive as they want to their own local copy; I don't see the role of the container at all in this case.

[ilinkuo]

@zackferraro I'm not proposing that Apps not be allowed to install plugins, but that they do not have absolute freedom to do so, and that it's the Container which is in a better position to decide what should or should not be installed because it has access to the bigger picture.

Giving each plugin its own instance solves a lot of incompatibility problems, but not all. I'll take an example that actually happened. An F2 App written by MOD added a jQuery scrollable plugin which broke our Dojo-based Container. How did it do that when our Container doesn't even use jQuery directly? The App jQuery plugin was written in the older jQuery style of being AMD-friendly but not quite AMD, and the AMD-compatibility part was incorrectly written. As a result, it broke the AMD loader whenever it ran, but would throw no error itself. We only managed to diagnose it by noticing that we had problems only when that F2 App was loaded, and the problems only occurred with the Apps which were loaded after it. This is an example where no one else was using the jQuery singleton, but it was nonetheless incompatible with the Container. It is for situations like these that I argue it is the Container which must be the final arbiter of what plugins are allowed.

Nonetheless, I almost agree with your statement

apps can be as destructive as they want to their own local copy;

To be precise, I agree when it comes to small modifications to F2, but not when it comes to plugins which load libraries, which could conceivably be heavyweight.

Let's look at a bad situation that could conceivably come up in the near future. The Container I'm working on can currently hold up to 20 F2 Apps, and it's constructed with Dojo, not jQuery. When F2 removes its dependency on jQuery, how are the apps which are written using jQuery going to get their jQuery? It's not going to be ok to allow them to directly load jQuery via AMD because there are so many versions to pick as well as hosts, so it would be best if there were an F2 plugin to load jQuery and make it available to the App. There might have to be several versions of the jQuery plugin because some jQuery plugins work with some versions and not others, and maybe also because jQuery 2.0+ dropped IE support. If all 20 F2 Apps came from different App providers, there is a potential of 20 different versions of a heavyweight F2-jQuery plugin, if each App provider writes their own jQuery plugin and is allowed complete freedom to install it.

And it doesn't have to be jQuery. You can take D3, YUI, ExtJS, Dojo, or any non-trivial library. The plugin freedom for a single App becomes a problem for the Container when multiplied by 20. Imagine 20 apps loading on a page and then after 20 manifest requests, the browser has to go out and fetch all the dependencies for each of the apps' plugins. That's the situation I'm in.

On the other hand, if the Container is able to restrict the use of plugins, then I could declare in a Container manifest (this doesn't exist yet, but it's something I would like to see) that my Container offers two jQuery version plugins, a D3 plugin, and a YUI plugin. These plugins I could minify and concatenate to be delivered in a single js file upon Container load. This is not something that can be done in the plugin model Chris proposed because the Container is completely out of the loop.

Now, to actually make this work, the F2 Spec would have to be modified to allow some kind of negotiation between Container and App, but this is really to early to talk about this since F2 isn't even AMD'ed yet.

[zackferraro]

What if App 1 needs jQueryPlugin1.1 to work or else it breaks, App 2 needs v1.2 or else it breaks, App 3 needs v1.3 or else it breaks... The container arbitration doesn't fix that. If you have 20 apps that all require a different version of jQueryPlugin to work and the container says, "You all get my version of the plugin," you have 19 broken apps. If you need 20 versions of jQueryPlugin on the page, then you need 20 versions of jQueryPlugin on the page.

I imagine that absurd case is not true, but if it were then that is what using the AMD pattern solves, if two apps "require" the same version of jQueryPlugin then whatever AMD library you're using only fetches it once. If more than one version of the plugin share a jQuery version "requirement", then jQuery is requested fewer times as well. The solution is not to try to subvert a plugin's dependencies by supplying it what you think they can handle. If I'm an app developer and I write my definition requiring jQueryPlugin v1.4 because I know what my app needs and I chose that version because of specific requirements, I don't want some container developer saying, "Your app doesn't work," and I have to figure out it is because they handicapped my app by forcibly ignoring my dependencies.

[ilinkuo]

@zackferraro Yes, AMD mitigates the problem but, in my opinion, not enough. If you load the dependency using "jquery" then you're relying on whatever version the Container makes available. If you load the dependency via an explicit url ", then there's a whole bunch of urls available and AMD is not going to prevent duplicate loading if the urls are different. If MOD, Company X, Company Y, Company Z all need jQuery, they're not going to coordinate and use the same urls, and the exact same version of jQuery could get reloaded from different urls, AMD or not.

On the other hand, if there were an official F2 registry of plugins, then the Apps needing the same version of jQuery plugin would be able to specify the plugin version needed in a common way, and, in that case, the use of AMD as the underlying load mechanism would indeed result in a benefit. This is an example of how coordination through F2 can reduce the duplicate loading problem.

The better long term solution, in my opinion, is to have the App manifest declare the version range of jQuery it is compatible with, and the Container provide its choice of version, but the spec doesn't support that. In this way, the Container serves as the hub of coordination rather than the F2 plugin registry. In this way, the App developer's choices are respected and there is no problem of

subvert a plugin's dependencies

With this arrangement, it is in the App's interest to verify compatibility with as many versions as possible so that it can be accommodated by as many Containers as possible.

What happens if the Container is unable to provide the requested dependency or the right version of it? My answer to this is what seems to get the goat of F2 App developers -- the App just doesn't load. This is the same behavior as when an AMD dependency goes unmet. And the reason for this is, to paraphrase your words, I don't want some cowboy app developer breaking my Container and I have to go through each App to figure out which one it is that's loading bad dependencies.

[zackferraro]

@ilinkuo For an official place for F2 plugins to reduce duplication, you yourself referenced https://github.com/OpenF2/F2Plugins just yesterday. That might not be the best long term solution, but it is a solution that currently exists.

The other points are not specifically plugin related, but at the risk of going off topic:

For versioning, there is nothing in the app manifest specifically for versioning, but once 1.4 comes out with custom script loaders the scripts array can be whatever you want it to be:

foreach(script in scripts) {
 if( typeof(script) === "string") // treat it like a URL for a script
 else {
   // script: { key: "myPlugin", version: ">=1.1 <2.0", fallbackUrl: "https://cdn.internet.net/myPlugin1.9.js" }
 }
}

Likewise you will probably get the ability to do whatever duplicitous dependency alterations you want; mapping https://cdn.internet.net/myPlugin1.9.js to https://cdn.ilinkuo.gov/F2AwesomePlugin2.8.js in a multitude of nefarious manners. If you consolidate your js files into one, then doing that mapping is something that is potentially possible depending on which AMD script loader you use; requirejs allows you to do various types of nested mappings.

I think the F2 group has a role in "strongly suggesting" conventions around that kind of thing such as "your scripts object needs to be an array and your loader has to have support for the elements being raw urls, and we recommend something like package.json for your version naming scheme".

[ilinkuo]

once 1.4 comes out with custom script loaders

This statement makes me sad because custom script loaders are, in my opinion, a large step away from App portability between Containers. I was opinion that the javascript loaders wars were over and that AMD won, but apparently OpenF2 doesn't think so. I'd much prefer a clear step towards preparing for AMD in F2 2.0 by saying that scripts should be an array like

["container/jQuery","container/d3/2.0"]

if using straightforward AMD.

A slightly better approach, in my opinion, which allows version matching and fallback urls would be to write a container AMD plugin to do the matching and fallback, with syntax such as

["container!jQuery!1.9+!http://code.jquery.com/jquery-2.1.0.min.js", "container!d3!>=2"]

There is an additional advantage of AMD-inspired approaches is that the Container can publish the plugins it supports as an array of AMD module IDs if the F2 spec agrees on a standard for naming F2 plugins and/or dependencies.

`["container/jQuery/1.9","container/jQuery/2.0",...]

mapping https://cdn.internet.net/myPlugin1.9.js to https://cdn.ilinkuo.gov/F2AwesomePlugin2.8.js

This is another problem that's already solved by AMD -- the dependency is referred to in the same way whether it is in separate files or minified, and I don't have to do an explicit mapping. There's no need to take this backward step, in my opinion.

we recommend something like package.json for your version naming scheme".

Would you all please consider instead abandoning the custom F2 manifest and choosing instead to replace it by an extension of package.json? That would be a major step towards compatibility with other parts of the javascript ecosystem such as npm.

[markhealey]

Hi all, very good and lengthy discussion happening here. I wanted to comment specifically on the note by @zackferraro:

once 1.4 comes out with custom script loaders

I'm not aware of anything new coming to F2 around custom script loaders. The 1.4.0 pull request —#142— doesn't include it as a to-do and we're keenly aware of the popularity of AMD-based dependency loading so not working on anything new. @zackferraro I'll catch up with you Friday as to what you're referring to.

@ilinkuo regarding the AppManifest vs. package.json question, we should open a separate Issue or use the new F2 v2 wiki to have a pros/cons discussion. The screaming popularity of npm, Chrome Apps/Extensions, etc indicate the package.json-inspired approach has been validated by the dev community and should absolutely be considered.

[zackferraro]

@markhealey I was just going off of https://github.com/OpenF2/F2/wiki/Roadmap. I don't have anything to submit for this.

@ilinkuo

I was opinion that the javascript loaders wars were over and that AMD won, but apparently OpenF2 doesn't think so.

I don't speak for OpenF2, I'm just an app developer and container developer. It is important to me that even if AMD is the core paradigm for script loading in F2, that the spec doesn't exclude people who don't want to use AMD. Regardless of how popular AMD and package.json are, they don't have 100% penetration and won't be around forever. F2 can either be a platform allowing people to use it how they like, or it can be a suite of pet technologies requiring a course correction whenever the ever changing internet changes again. Otherwise when--not if--the next great thing comes along F2 will be instantly become obsolete and left scrambling to catch up.

[markhealey]

Thanks @zackferraro. I see what happened, I just updated that item to be more specific in the roadmap.

[ilinkuo]

@zackferraro, for my opinion, please see this comment

[ilinkuo]

For an official place for F2 plugins to reduce duplication, you yourself referenced https://github.com/OpenF2/F2Plugins just yesterday. That might not be the best long term solution, but it is a solution that currently exists.

@zackferraro, it's a solution in name only, unfortunately.

First of all, no one actually uses it. Though you've written plugins, they're not there. Thus, it's an abandoned repo. F2 must provide guidance and actively encourage contribution in order to change this.

Secondly, once a plugin is published there, it's quite likely that when I install it, it won't work for me because the plugin developer used a different version of F2, but I won't find this out until I actually try to use the plugin. This will increase the frustration and lead to loss of confidence in the plugin repo. A better experience is one where the plugin installation process fails immediately because the plugin is the wrong version.

Thirdly, there are no tests for the plugins and there are no Travis-CI jobs running against the plugins to ensure the tests pass with each F2 version. So if there is version compatibility information in the docs, then it's not enforced or checked.

[brianbaker]

Unfortunately, many of the sites and developers using F2 aren't able to contribute code or even comments for legal reasons. That's just fact of working in the industry and a hurdle that we'll have to figure out how to get over so there can be viable plugin contribution and approach. Lets take a step back and remember that this thread is to discuss how to handle plugins - I think everyone involved in the thread is in agreement that updates need to be made.

I think the plugin aspect of F2 should be solved within the context of a v2. At this point, it doesn't seem fruitful to work towards a v1 solution if the way F2 and apps are written has to change for v2. Let me try and summarize the lengthy prose above so that my feeble mind can see the requirements in one place.

Plugins Requirements:

• Should be compatible with AMD
• Allow namespacing onto the F2 object
• Allow key components/internals of F2 to be swapped out
• Check version compatibility
• Allow multiple versions (without conflicts)
• Allow multiple instances of F2 so that apps and containers can have their own plugins without effecting each other. There was a concern about memory usage in a pattern that provided multiple instances of F2, though.
• Must be unit tested

Sorry if I missed any requirements. Is there a better way (via GitHub or something) that we can have a community editable list of items/requirements?

[markhealey]

Unfortunately, many of the sites and developers using F2 aren't able to contribute code or even comments for legal reasons.

Specifically it is client confidentiality and NDAs preventing some (but not all) developers from contributing plugins. Internally at MOD we should look to find ways to explore opportunities this year.

Is there a better way (via GitHub or something) that we can have a community editable list of items/requirements?

I think the best approach is to do what you've done but lift the bullets up to the top in an updated Issue description. As maintainers we have this responsibility. Additionally we should copy the bullets across to the v2 wiki and close this Issue since it feels like we've covered the requirements/issues. Remember this is just a proposal so when we actually have a branch and some v2 code we can pick the conversation up again if needed.

Thanks everyone and @brianbaker for summarizing and focusing this back.