Koa as a generic middleware solution?

[aewing]

The repository description reads:

Expressive middleware for node.js using ES2017 async functions

The first human-readable line of the readme reads:

Expressive HTTP middleware framework for node.js to make web applications and APIs more enjoyable to write.

Essentially the package is a class wrapper around koa-compose, allowing the composition and execution of a middleware stack on request from an HTTP server.

But why only HTTP? Is this pattern not applicable to many request+response services?

Why?

Middleware offers a functional solution to working with requests+responses services, giving developers a tool to reuse common context manipulation & comparison and assert the cleanliness of context (by early stack termination), making it an ideal container for business logic. The current nature of Koa makes it inapplicable to other request+response services common to a stack that might contain Koa and benefit from standardized request+response and common logic for other services.

Why Koa?

Admittedly, there are several other middleware composition alternatives available, including koa-compose, that could just as easily be wrapped in a class and attached to an emitter. Sure, this is true, but the active nature of the Koa ecosystem makes it an ideal platform to offer standardized middleware support for request+response environments. The value of being able to offer some level of standardization to various environments with a functional component that is so flexible is very appealing to me, and I would hope to developers at large.

What would need to change?

As I see it, Koa could easily become service-agnostic while maintaining it's existing functionality as an HTTP server utility in several ways. Some less awesome than others.

Constructor Configuration

The easiest & cleanest option, in my opinion, is to enable a constructor configuration supporting the following or similar properties:

new Koa({
  provider: (callback, args) => { // Callback being the output of this.callback() from listen()
    const server = http.createServer(callback); // Allows one to connect Koa to the emitter
    return server.listen(...args); // Args provided to the listen() method as usual
  },
  onRequest: (args) => { // Agnostic context assembly, provided all args from callback
    return {
      // Service-specific context assembly
      // Implement createContext by default
      // 404 defaults, etc... previously handled in createContext
      // Perhaps an internal/immutable context could be created here and shared with the response?
    };
  },
  onResponse: (ctx) => { // One last method to call when the stack is finished executing
    // for HTTP, this would essentially be the respond() method in lib/application.js
  }
})

This solution assumes that the call to createContext in the callback method is replaced with a reference to onRequest, which would return the initial context object. It also assumes that the handleRequest method pass the context to onResponse after stack execution (which is where the statusCode would be assumed, errors handled, and on-finished implemented for HTTP).

Default parameters could be assigned to this configuration with spread assignment, allowing the class to work out of the box with HTTP support from an external package. Perhaps the class wrapper could/should be abstracted to another package, i.e. koa-app.

I would even consider removing onRequest and onResponse altogether and providing the parameters given to the callback to the middleware, allowing middleware to assemble the request and become responsible for the response.

At any rate, I just wanted to share my thoughts on the matter and see what the community feedback is. If this is a direction that the community is open to exploring, I would be happy to submit a PR.

[aewing]

One of many potential use cases (in the near future):

• Foo would like to offer a restish interface via a Redis pubsub interface, a websocket, and an HTTP server.
• Foo will be using the same ORM models and business logic to complete this task for each service.
• Foo decides to use the Koa proposed in this specification.
• Foo finds and installs Koa context plugins for redis pubsub, websockets, and HTTP.
• Foo finds a Koa router plugin that supports all of these context plugins.
• Foo implements a single router that executes his business logic in a predictable fashion across the three service.
• Foo wins.

[jbielick]

This is a really well-written proposal/question. Thanks for the work that went into it. I thought I'd share some thoughts to further the conversation.

Admittedly, there are several other middleware composition alternatives available, including koa-compose, that could just as easily be wrapped in a class and attached to an emitter.

Couldn't agree more. To me, koa-compose is the most sensible abstraction; context, and the rest are the implementation details. It could be argued that Koa's popularity and elegance is derived from its diligent focus on doing no more and no less than it should. I tend to believe abstractions don't hit their mark until there are more than a couple concrete examples of what needs to be abstracted.

I freely admit I don't have experience writing a redis pub/sub or websockets framework (one that needs middleware or otherwise), but when inspecting an example of each of these would the author really list the functionality of Koa as a needed abstraction?

When I needed a middleware solution for a client library I was writing, koa-compose was the first thing that came to mind, because I needed middleware and already had the rest of the intention-and-project-specific code/scaffolding already written. koa would have just been overkill.

Sure, this is true, but the active nature of the Koa ecosystem makes it an ideal platform to offer standardized middleware support for request+response environments.

When I read this I heard "koa has done a great job of addressing a single and specific need for web developers. I want other usages of request+response environments to be as successful".

I fully understand that's not what's being described here, but I'm stuck with the thought that express was a beautiful middleware API (for its time) and has existed for quite some time, but no abstraction of it has really been compelling. Plenty of libraries were authored during its lifetime that fit the use case described here, but very few come to mind as something that could benefit from the entirety of the express application class. The middleware composition / execution would be the best part to abstract—which leaves me back at "koa-compose is the answer".

This topic reminds me of the node.js convention of callback arguments (err, result). "Every lib should do it this way" was the cry, but the part that everyone needs to do is very small and the tool to help everyone do what we're talking about here might be koa-compose.

I think the amount of code that fits in between koa-compose and koa (e.g.koa-the-abstract-middleware-registry-and-invoker-and-event-emitter) is very small and would be difficult for anyone to write with a clear understanding of what ought and ought not be written in that layer of abstraction. I also think it would be something a library author would have opinions about that are very context-dependent and not necessarily abstract.

[aewing]

I haven't forgotten about this thread. I have been actively prototyping an architecture similar to the one I've described here in attempt to answer some of these questions for myself. I will be returning with answers shortly. In the meantime, you can view my progress at https://github.com/collaboratory/emitterware

[EduardoRFS]

That proposal is amazing, and make me see a lot of insights, the most obvious case of using it ... websockets. Turning Koa agnostic allow us to just create an middleware make the API compatible with a lot of middlewares who is already in use.

Like an REST API over websockets, we just implement the Koa API, mapping to an websocket library, and now we serve a lot of things over websockets, without even the middleware knowing what is happening. Using the same library who make "koa over websockets" we can serve files using koa-static.

And all of that without breaking an single line of code, we just need to create an way to overwrite default Koa behaviors or perhaps like @aewing said move actual koa implementation to another package and that package is used by default, but we can just replace that package.

It is different from koa-compose on keeping the actual API, just like if Koa turn into an API specification with an http official implementation

[aewing]

Exactly, @EduardoRFS -- I have seen some pretty remarkable success with websockets and message queues operating in a RESTish fashion. I'm getting really close to having some conclusive examples and findings to bring back to the table (I've been working on this on weekends and free time).

[aewing]

One of the side effects to this approach is the ability to run various daemons from within a single binary while sharing database connections and middleware. In my emitterware repository that I linked above, I have established an "app" package that allows a developer to register various "providers" and add middleware to a specific provider or the global provider stack. Middleware can be weighted to determine priority in the various stacks. This requires me to think a bit differently about how and when I load my middleware, but it allows me to reuse common logic across the various services I utilize (session middleware that doesn't care which service it runs on). So far I am really enjoying building and working with the emitterware ecosystem. I look forward to soon turning my experiences into a more meaningful proposal.