Allow handlers to return a CompletionStage

[mszajna]

The proposal is to extend Ring's 1-arity handler contract to allow returning of a java.util.concurrent.CompletionStage-wrapped response map, as an alternative to just returning a response map.

(defn a-handler [request]
  (if tuesday? ; handler is free to return a sync or an async response
    (java.util.concurrent.CompletableFuture/completedFuture {:status 200})
    {:status 400}))

Why?

To support asynchronous flows, Ring offers a 3-arity variant of the handler contract that accepts extra arguments of successful and exceptional callbacks. This is sometimes referred to as continuation-passing style (CPS). Technically, it's a perfectly capable solution that supports flows of arbitrary complexity. Writing correct CPS-based software is very much possible but in practice, really hard.

The documentation doesn't formalise the contract. It's unclear what are the semantics of invoking both callbacks or calling either multiple times. In the domain of HTTP request/response one could infer that neither makes much sense. Yet inadvertently, this does happen in practice. Manual callback invocations are hard to reason about and therefore prone to errors. The linked issue explores a possible restriction on callbacks not to throw, to make that reasoning easier. Still, with no way to enforce it, the model is fragile. The burden of compliance and consequence of third party code non-compliance is left entirely to the developer.

This is not a new problem, unique to Clojure or Ring. Other languages and platforms have been through a similar journey. The solution seems to converge at a monadic eventual value abstraction (JS, Scala, Kotlin, Rust, Haskell), such as CompletionStage - the interface behind CompletableFuture. It mimics closely the semantics of a call-stack based execution (that's what makes async/await transformation possible). The monadic interface shepherds, if not forces the developer into writing correct code with proper error propagation and at-most-once flow execution.

Clojure is built around evaluating expressions. Users have developed understanding and intuition for working with them. The 3-arity contract subverts that with functions that don't evaluate to anything useful and the need to imperatively communicate continuation. CompletableFuture, while imperative in many ways, promotes the use of combinators that guarantee the code executes once and exceptions won't get lost.

Opt-in async

The proposal defines the return value as either a regular response map or a CompletionStage eventually delivering one. Arguably, a stricter contract with purely CompletionStage return values would be simpler and handier for middleware to interact with. Still, I believe the optional async to be a good trade-off.

In a single app, not all flows are async. Having to decide upfront is a tough choice. A contract that supports both makes it easy to start with synchronous code and migrate to async when and where the performance benefits are worth it.

The other point is that this approach allows (but not forces) library authors to support both models in a single function.

Compatibility

Existing 1-arity handlers are fully compatible with this proposal, i.e. they continue to work fine even in a middleware that uses CompletionStage. Existing 1-arity middleware limited to request altering and pre-handler side effects is also fully compatible, i.e. it continues to work with both regular and CompletionStage-returning handlers. All other 1-arity middleware can be easily made backward-compatible, i.e. if the wrapped handler returns a map the middleware returns a map too.

An easy path of adoption

Having middleware anticipate a response map or a CompletionStage and branch the logic on the return value is not particularly convenient. To make it efficient Ring could ship with some basic tooling. For response processing I propose bind, inspired by Promise.then, that preserves compatibility with synchronous Ring. This, together with the forward-compatible request-processing middleware cover all but one use case of ring.middleware.*. It seems to fit the model of making common cases easy and less common ones possible.

(defn bind
  "Run f on a possibly eventual response map, expecting to receive another
   possibly eventual response map. If both are immediete responses returns
   a response map. Otherwise returns an eventual response map."
  [response f & args]
  (if (async? response)
    (.thenCompose ^CompletionStage response
                  (java-fn [response-map] ; hypothetical macro
                    (let [response' (apply f response-map args)]
                      (if (async? response')
                        response'
                        (CompletableFuture/completedFuture response')))))
    (apply f response args)))

(defn wrap-not-modified [handler]
  (fn [request]
    (-> (handler request)
-       (not-modified-response request)))
+       (bind not-modified-response request)))

Why CompletionStage?

The argumentation so far has largely been unrelated to a concrete implementation of the eventual value abstraction. I would like to push the case that CompletionStage, the interface behind CompletableFuture is the best fit for Ring.

CompletionStage was first introduced with Java 8 in 2014. According to this survey, a vast majority of 99% of Clojure developers target that or later version. Java 8 has also become a requirement as of Clojure 1.10, released in Dec 2018 and nearing 90% adoption according to the same survey. The interface is available for most Clojure developers with no extra dependencies.

Clojure is a hosted language. It builds on top of JVM and its ecosystem of libraries. Adopting a platform-supplied abstraction is a natural thing to do. The CompletableFuture has gained a significant adoption (reactor-netty, Scala, Kotlin). Java 11 ships with a built in HTTP client based on it, wrapped into clj-http-like client by hato.

CompletionStage is far from perfect. It ships with a myriad of similar methods. Using it with Clojure directly is particularly awkward with the need to reify java.util.function.* interfaces. This is hopefully going to eventually get addressed. The awkwardness can be alleviated by libraries like promesa. The important point is that the guarantees are upheld regardless.

Why not core.async?

Some Clojure developers want to believe that core.async is the async abstraction of the language. I can empathise with this sentiment as I am, too, disappointed with community's failure to elect a single, commonly accepted abstraction for eventual values. But core.async isn't one.

As discussed in this thread, channels lack error semantics. Unhandled exceptions in go blocks bubble into non-existence. It's not a flaw in core.async by any means. It's just an implication of adopting the model of Communicating Sequential Processes (CSP, not to be mixed with CPS). This model does not relate to the synchronous Ring very well.

Why not a protocol?

If Ring was a library that solely consumes eventual values, a protocol would make sense, as the surface of that protocol would be limited. This is not the case however. Ring is a contract that many developers write libraries against, with eventual values being both consumed and returned. There is a lot of scenarios library authors might want to accomplish including combining eventual values, racing, parallel execution and more. Maintaining a comprehensive eventual value abstraction is beyond the scope of this library.

Proof of concept

This commit demonstrates the updates necessary to make this proposal happen. There's surprisingly little in there. As you may notice, most middleware is neatly handled with bind. The error handling scenario is a little more involved.

Error handling middleware

While the abstraction of eventual value helps to make sure exceptions propagate correctly, handling them correctly requires processing the rejected CompletionStage and any bubbling exceptions thrown by handler within the thread. One example can be found in the fork.

If you're not concerned about backward compatibility with synchronous ring, the simplest idiom is to execute the handler in the context of a CompletableFuture and then .handle

(-> (CompletableFuture/completedFuture request)
    (bind handler)
    (.handle (reify BiFunction
               (apply [_ response-map exception]
                 (if exception ..handle.. response-map)))))

Middleware that needs finally-like semantics is doable, thanks to CompletionStage's at-most-once semantics.

Existing post-response side-effecting middleware

Most existing middleware is either forward compatible with this proposal or is obviously incompatible, ie. an attempt to use it would result in an exception. There is a niche of middleware though, that continue to work in a faulty manner. In particular, any middleware that expects to run certain side-effects after the processing has finished (as in finally) will run those immediately after the handler has returned the CompletionStage. In some cases the deferred logic contained within will fail visibly because of that, alerting the author of the problem, in others the problem may remain unnoticed. While the issue is real and deserves appropriate documentation, I expect it to be rare.

[weavejester]

Thanks for the proposal. Rather than going straight to a PR on Ring itself, I think it would be better to begin with a middleware interface that converts handlers that return a CompletionStage into the 3-arity form Ring currently supports. This would allow developers to test it out without making any corresponding change to Ring, and give us some idea of how useful it would be.

[mszajna]

Here's a gist with conversions between 3-arity and this proposal. There's also an adapter for middleware allowing one to code against this proposal across entire stack, not just at the edges. I might turn it into a proper project later.