This addresses a long-considered need in DocumentRouter: the ability to chain them.
Description
A common use case would be connecting a DocumentRouter that does some transformation (e.g. dark subtraction) to a DocumentRouter that does visualization or serialization. Currently, the only way to do this is:
The relationship between documents received and documents emitted does not have to be 1:1. Note that the base class does not callemit internally at all. It is up to subclasses to define whether and when to emit. For example, Serializer classes and classes that produce visualizations are generally "sinks" and have no need to incur the overhead of emitting the documents to a downstream node. Some applications (example) may one to emit multiple documents when they receive a single document.
This emits straightforwardly to at most one downstream node. For more complex behavior, like fanning out to multiple nodes, handling backpressure, and other considerations, the user should engage a more complex library like streamz by registering a Stream node with emit. By accepting this one additional layer of indirection, we keep these considerations separate from event-model.
Alternatives Considered
We could leave it up to subclasses to add emit but a standard emit method and __init__ parameter
This addresses a long-considered need in DocumentRouter: the ability to chain them.
Description
A common use case would be connecting a DocumentRouter that does some transformation (e.g. dark subtraction) to a DocumentRouter that does visualization or serialization. Currently, the only way to do this is:
which is verbose and error-prone. This PR enables:
Design Considerations
emit
internally at all. It is up to subclasses to define whether and when to emit. For example,Serializer
classes and classes that produce visualizations are generally "sinks" and have no need to incur the overhead of emitting the documents to a downstream node. Some applications (example) may one to emit multiple documents when they receive a single document.emit
. By accepting this one additional layer of indirection, we keep these considerations separate from event-model.Alternatives Considered
We could leave it up to subclasses to add
emit
but a standardemit
method and__init__
parameter