Reaching out to DocPad

[shama]

DocPad is a really neat project and has matured a great deal. I think DocPad plugins have a lot in common with task spec being discussed here. It would be great to reach out to @balupton to have him weigh in and include DocPad in this spec discussion.

[sindresorhus]

:+1:

Previous discussion: https://github.com/gruntjs/grunt/issues/149

[balupton]

Thanks for inviting me to participate.

The re-use between docpad plugins and grunt tasks or whatever other node runner is one that has weighed a lot of consideration for us as well.

---

There are a few things that make DocPad plugins a bit different from the current node-tasks specification:

1. We load plugins automatically by scanning the node_modules directory, no need to alter a configuration file to have plugins execute
2. Plugins stay alive for the duration of the docpad process (#6)
3. Multiple configuration sources (e.g. instance configuration, user configuration, environment configurations) are merged together
4. When a change is made to the project's configuration, we just re-apply the configuration on the plugin while it is already loaded (rather than destroying and creating it again) - not sure if this is intended for parseConfig or not
5. We have no setup event besides the initial constructor, and no teardown event - haven't needed them yet, happy to have them
6. The plugins are event based (#13) using https://github.com/bevry/event-emitter-grouped
7. Event flow is executed serially, with event listeners being executed either asynchronously or synchronously based on whether or not they accept the next argument (the second argument to the event listener)
8. Plugins have access to the docpad instance which contains the database of all the files they'll be interacting with

---

However, for the future we'll also be tackling the following:

Cross-plugin communication. Soon we'll be moving the core actions (generate, server, watch) into their own plugins, as such we will need a way for the plugins to be able to communicate with each other. For instance the livereload plugin needs to be able to get the server plugin's server instance, and for the watch plugin to interact with the generate plugin when files change, we'll even be taking this as far as hopefully moving out even the docpad database stuff into it's own plugin allowing other plugins to decide how to save changes or import files. This is quite a challenge, as say should the livereload plugin call this.docpad.getSever() as it does now (which would be injected by the server plugin) or should it call this.emit('getServer',function(err,server){}) to get the server (an event exposed by the server plugin).

This is also were I see the advantage of such a unified architecture coming in, as with modularization, we could make docpad just a series of tasks that cross-communicate and co-depend on each other - e.g. install the task for docpad database, docpad cli, docpad generate, docpad markdown, etc - and then be able to introspect what the available events in your project are - okay we know have the getServer, serverBefore, serverAfter events thanks to the docpad server plugin, etc - so all plugins do is expose and listen to events for each other - this seems to be what marak squire's has in mind with big to some extent - though perhaps this is all a bit more resource like than task like though, as docpad plugins are more like big's resources, where they handle a specific area of functionality, rather than just performing one task in a forget-then-die way - this distinction between resource and task seems to be an issue that grunt is facing now (#6), things like compilation is a task, but things like livereload is a resource, treating everything as a resource in the way docpad does appears to accomodate for both resources and tasks quite well

Event order. Already we are starting to see some document renderers conflict in their execution order (e.g. plugin a must execute its listener for event blah before plugin b). The issue here is should it be the plugin's responsibility to know of it's incompatibilities (e.g. plugin a specifies: eventOrder: blah: runBefore: ['b']) or that of the project's configuration (e.g. project config has: eventOrder: blah: ['a','b'] - with any unlisted plugin that has event blag going in any order) - I think the latter, as incompatibilities are a blind spot, you can never predict what outside your blackbox will happen, and as the project knows all, it should know the correct order.

---

So that's were we are at, whether or not any of it is a good idea or not I don't know, I could be wrong, often I am, so happy to explore the options :) It seems that the big question here is whether or not node-tasks wants to become more node-resources to facilitate things like big, docpad, and other long running tasks etc, rather than focusing solely on the live then die and isolated tasks? as the two options seem to be polar opposites in regard to implementation from a first quick evaluation

[shama]

Awesome. I really appreciate you taking the time to so throughly review and weigh in here. Your task vs resource remarks make a good point. There are some grunt plugins that are awkward as tasks. Until now I haven't put much consideration into treating those plugins as something other than a task. So thank you for that and thanks for the links to Big as well.

I think a lot of the current comparisons you made could be handled by each of the task/resource runners and still keep the plugins generic. Although not having thought as far as you have about cross-plugin communication. The biggest blocker I see is the need to pass an instance of DocPad (or any other task/resource runner). As if any plugin requires an instance of a task/resource runner then it would be locked to that task/resource runner. Would it be feasible for a DocPad plugin to operate without an instance passed?

Thanks again and sorry for not responding to all your remarks.

[balupton]

Would it be feasible for a DocPad plugin to operate without an instance passed?

Yep, with the cross-plugin stuff that'll allow us to make it so that instead of passing the instance of the runner with the constructor, instead it is retrieved from an event call or whatever else we use as the standard for cross-plugin communication.

Thanks again and sorry for not responding to all your remarks.

No worries, keen to help and see where this goes :) Thanks.

[tkellen]

Closing this in favor of the current ongoing discussions. Thanks for your input Ben!