Open kachayev opened 5 years ago
The short-circuiting also allows us to do something like this:
(let [r-a (query-a)
r-b (query-b)]
(d/connect r-a r-b)
r-b)
This means that r-b
represents the first query to complete, and if r-a
comes in first we'd ideally like query-b
to stop doing unnecessary work. I think it's very possible no one's doing this, and most of the use cases reflect the sort of behavior your API encodes, but I definitely don't want to have a special case cancelled?
predicate, because realized?
is really the important question to be asking.
I've been thinking about cancelled?
not in terms of another realized?
predicated but mostly as a way for the user to distinguish error from cancellation: let's say I want to log errors but I don't need to do log anything if I was simply canceled (at least that's not my job to check why that happens, someone who's responsible for cancellation should do this).
I think they don't use d/connect
because of 2 reasons:
The approach is not that obvious. Even having some experience with the library.
In practice you probably need to rely on the results of both queries, so you will end up doing d/zip
or d/alt
at some point.
Regarding "cancellable" d/zip
... I can think of at least 2 different use cases that I have in practice:
Cancel the result of d/zip
as a way to say "I don't need this thing any longer". I would want the library to take about canceling all related deferreds.
"Fail-fast" version of d/zip
: cancel all related deferreds when one of them failed. It's a pretty common practical situation: I fired a bunch of queries, one of them failed (for any reason), I know that I have no way to finish my computation, so I would like to stop the entire graph of computations right now.
I think those exhaust the list of potential use cases which are not covered by short-circuiting of chained listeners. But maybe I'm missing something.
I think those exhaust the list of potential use cases which are not covered by short-circuiting of chained listeners. But maybe I'm missing something.
As I mentioned in #166 current implementation doesn't cover even chain use cases, for example, nested chains.
@prepor This one?
(->
(deferred/chain
(deferred/chain
(deferred/future (Thread/sleep 2000))
(fn [_]
(prn "----FINISH")
:finished))
(fn [_] :finished))
(deferred/timeout! 1000)
(deref))
In this example printing of "FINISH"
is expected as d/timeout!
in case of the error short-circuits all subsequent listeners. E.g.
(->
(d/chain
(d/chain
(d/future (Thread/sleep 2000))
(fn [_]
(prn "----FINISH")
:finished))
(fn [_] :finished))
(d/timeout! 1000)
(d/chain (fn [_] (prn "after timeout")))
(deref))
"after timeout"
will never be printed.
The idiomatic way to "cancel" deferred in Manifold is to put an error value in order to short-circuit all chained listeners. Recently we had quite a few questions around this specific functionality... So, I wonder if it's better to have a public API in place to deal with cancellation? E.g.
We also need to document that cancellation of
d/future
ord/future-with
does not interrupt the underlying thread (which would be expected because offuture-cancel
semantic from Clojure core).More advanced thins in terms of cancellations:
d/zip
andd/alt
that propagate cancellation to underlying deferreds.It might be simple for 2 mentioned functions... but maybe we can think of a more general approach/solution where we can manually specify (or automatically derive?) a graph of connections between different deferred to use it to propagate
CancelledExceptions
properly?Keeping in mind previous item... should we treat
TimeoutException
as a cancellation?Self-referencial
cancelled?
checker ford/future
. That's arguable, but often times I need this:I'm not sure about this specific feature as it undercovers underlying mechanics to some extent. It's still doable by introducing a separate deferred in a lexical scope, but this approach might not be obvious for beginners. So, at least it should be well-documented.
@ztellman WDYT?