search

reactive-ipc / reactive-ipc-jvm

Reactive IPC for the JVM
Apache License 2.0
55 stars 13 forks source link

Need of the `Buffer` abstraction #10

Open NiteshKant opened 9 years ago

NiteshKant commented 9 years ago

@jbrisbin mentions the intention of this abstraction here as follows:

I also added a Buffer abstraction in core. I put it in the javadoc but it should be noted that a Buffer in ripc terms does not mean just a variation on byte[] or ByteBuffer. The only (partial) implementation that exists right now is one based on Netty's ByteBuf, but the Buffer abstraction itself doesn't care what the underlying data actually is. It could be a Buffer or a Buffer. This will provide simple read-and-convert functionality for common things like length-field-based encoding that uses Snappy or Zip compression, etc... that's going to be nearly identical for all use cases.

I am unclear about why would we have to provide data encoding/decoding via a common interface agnostic of networking libraries. I would imagine the codecs naturally fitting with a networking library interface and general length-field-based codecs are already available with netty.

Currently, this Buffer is a part of the Connection as:

public interface Connection<B> extends Publisher<Buffer<B>>

This means that everything read from a connection is always wrapped in this Buffer. Is this a valid contract? IMO it is too opinionated.

jbrisbin commented 9 years ago

We don't need a separate module for an interface that extends Publisher<T>, we could simply expose that directly from a wrapper around Netty.

We had a discussion with some members of our team around this and a point @rstoyanchev brought up is a good one: if we're building a generic and reusable core, then our intention is that people not use the Netty API directly. Otherwise why abstract it at all and why go to the trouble of splitting anything up? If we're not going to have common and reusable functionality in any of these other components, then this project itself has no purpose. You can easily create an RS implementation right on top of Netty and simply expose the RS interfaces instead of the Netty ones. There is no need for other modules and other levels of abstraction if that's the case.

IMO we should not assume that all codecs are configured on the Netty pipeline and that the message being passed to the RS pipeline is one of the output type by the Netty codec and not the "lowest common denominator". This would immediately preclude the use of any implementation other than Netty unless it also provided its own, proprietary way of doing the common task of requesting that a chunk of bytes (which it would never access directly since it would get that through the Buffer abstraction) be turned into a useful object (which it would also never know about, nor touch, other than to pass it through to the next component in the pipeline). Without some kind of wrapper abstraction to represent the incoming data, there is simply no way for a generic component to interact with it other than to cast it to a real type, which could only be done in user code or in the implementation module.

Again we're back to the issue of: why bother with this exercise at all if we aren't going to abstract any of this "common" functionality away from Netty? It can't really be all that common if we are depending on Netty itself to do every bit of heavy lifting, from dispatching signals to decoding data.