search

haxetink / tink_streams

Streams from the future. With lasers, of course ... whoaaaaa!!!!
The Unlicense
12 stars 9 forks source link

Future version of `depleted` #13

Open kevinresol opened 7 years ago

kevinresol commented 7 years ago

Currently depleted is a Bool value indicating the current stream is depleted. So essentially only an Empty and its derivatives (Future, Compound, etc) will have depleted = true. (Or it should be more accurately named as isEmpty...)

While that is useful for code which iterates the stream, it isn't very useful for code that "observe" the stream. So I would like to suggest adding a property of type Future<Outcome<Noise, Error>> which when happens it means that "the stream's end has been observed". or in other words we can now iterate it in a sync manner.

This is different from iterating the stream myself and handle the result. Because the proposed future remains lazy until "someone else" iterates the stream to an end.

I think of this idea when I was trying to find the right moment to close the socket in tink_tcp.

back2dos commented 7 years ago

Yeah, I did some really wacky stuff with that for node: https://github.com/haxetink/tink_tcp/blob/pure/src/tink/tcp/nodejs/NodejsConnector.hx#L25

In hindsight I wonder whether that doesn't complicate things. I think that basically the socket should be closed from your end when you end your outgoing stream. When you don't want the incoming stream to stop, don't end the outgoing one (all of which is the handler's responsibility). Nice and clean. No allowHalfOpen and other weirdness.

That said, it has two issues:

  1. If you don't think about it, you may inadvertently chop off the incoming stream - although I think people could get used to that and honestly, who really does raw TCP anyway ...
  2. Protocols that aren't clearly separated from the transport layer would be impossible to implement. Assume that the remote end waits for your outbound stream to end before it writes its last message. It's clunky and horrible and wrong, but I wouldn't be surprised if it exists out in the wild.

Let's think about this. This exterior observation of internal state change doesn't fit too well with the idea of these streams that try really hard to pretend they are pure ^^ If we can find a nice way to live without it, I would very much prefer that. If not, then we'll find a way to add it.

I wonder though: can you think of other applications for this? It's kind of special about TCP that you have two so strongly couple streams.

kevinresol commented 7 years ago

I dunno, this is mainly because tcp handler's definition doesn't seem to fit well on blocking targets. Because it is Incoming->Outgoing, and we have to handle the incoming stream before returning the outgoing one. But that will block forever because you never send out anything. I am not sure what is the correct way to use it.

I realize my proposal here won't fix the problem either.

back2dos commented 7 years ago

Hmm. I think being blocking or not should not be that important. In theory input.parseStream(commandParser).map(execCommand).map(serializeResponse) creates a lazy outgoing stream of chunks that would be driven from by the remote end reading the responses.

One of the problems right now is that if you set a socket to be non-blocking without runloop, then it runs into a stack overflow with empty chunks.

kevinresol commented 4 years ago

With https://github.com/haxetink/tink_tcp/issues/5 we probably don't need allowHalfOpen?