multiple response requests `Stream`, `Chunked` should be one 'Multiple'

[aricart]

The concept of Stream, Chunked, Singleton has some clarification - singleton is obviated, as that is the standard pattern.

Chunked is intended for segmenting a large piece of data into multiple messages, the requestor can then catenate the messages and recreate the original data. The last message is followed by an empty sentinel message.

Streams are multiple messages. Possibly no end, or completed by an empty sentinel message.

If ultimately, the end of the stream of messages is signaled by a sentinel, then from an API perspective there's no difference. As a consumer of the service, the client already knows what to do. The clarification would be that when the subscription closes (the last message is received), the client could event that information to perform some other operation (in the case of chunked, join the parts).

[derekcollison]

They are very different and should be treated as such.

Streamed messages can be held in memory, they are just NATS messages. Signal with EOF (empty message).

Chunked represents a single thing that had to be chunked up and may be very large, say 128GB. So we need to allow streaming and such to files etc for receiving a chunked response as needed.

[matthiashanel]

@derekcollison @aricart , the only possible difference I see is that with Chunked - no matter how you design your chunking protocol - you can't afford to miss a message and thus cleanup should happen as soon as the interest for the response goes away. With a stream you may miss a message, that's why I would require a service time or (timeout time) to be specified with it.

An EOF message can be lost too and thus can't be relied on to clean up state.

I'd further consider two clients sending a request pointing at the same response subject. By which EOF signal would we want to go by? This should be the clients/users demission as signaled by ... unsubscribing

For RTT measurements I would count interest going away as RTT measurement failure and count it accordingly.

Non singleton is not yet documented, thus we can still change things: if the signal of the response going away does not exist due to inbox wildcards maybe it needs to be added back in.

One disconnect remains: The publisher does not know when the subscriber is gone.

The next is looking ahead: We suffer from this in our own apis.

Jetstream pull based consumer is such an example. (so is streaming) The next message can be lost somewhere in the network. The nats connection does not necessarily go down when this is the case. So now we are forced to pick timeouts:

• too small causing traffic when there are no new messages.
  • while being disconnected pollute the reconnect buffer that might be better to hold ack or next messages.
• too large, then we'll recover very slowly from message loss in the network.

Our customers will face these same issues when writing services, specifically, with chunked or stream response types. Despite this being smth. the customer programs, we'll get to deal with this...

Why not offer an advisory where you can subscribe to the state changes of subscriptions and thus get a sense what our "logical" connection as expressed by subscriptions looks like. If we added the ability of nats server to subscribe to subscriptions: $INTEREST.<some subject/possibly an inbox/reply subject>

1. When subscribing I get a +UP when the subscription exists.
2. get a +DOWN when it does not.
3. If $INTEREST. exists (got a DOWN already) and now is added, we send a +UP.

Disconnect by my application resends it's subscriptions, thus we'll find ourselves in situation 1 or 2. Disconnect by a node in the network causes interest to go away -> situation 2. Reconnect by a node in the network causes interest to re appear -> situation 3.

I believe when done right we could use this for jetstream and our customers could find this useful too.

[derekcollison]

You need to be able to detect loss with either streamed or chunked. Streamed is optional but if they all transition a state I need them all too, so same problem.

There will be an expected response threshold that can be added to a service export that can both signal the system and a requestor.

If someone sends two requests with same reply that is a misconfiguration. I am not worried about that.

Interest based cleanup is optimistic can never be relied upon in totality, at least as the system exists today. We will always need to cleanup when response data structures exist past the response threshold.

Having a publisher understand when a subscribe is gone I can see as useful for large chunked response interactions. TBD for sure.

Interest advisories are hard and can not be depended on, painful lesson learned back in the RV days. However, with JetStream code a server knows if a message has been delivered, so might be possible to piggy back that and send that back to the publisher.. More to think on there but I think this is a much better approach.

We had a long discussion early this am on how all of these things are coming together. JetStream, Streamed and Chunked responses and the desire for generic headers in base NATS.

[matthiashanel]

I'd rather see the system become better at interest based cleanup than adding eof.

Stream or chunked is for the service exporter to express if it's response data is idempotent or not. The customer needs to write logic to detect issues if it's non idempotent. The customer code also decides if and when it has enough and unsubscribes.

The way our apis are, when using multiple responses, you have to have created a subscriber yourself and thus unsubscribe has to be called anyway, or subscriber are leaked.

It the client wants an empty message to mean signal the end of my stream, the receiving code should interpret it this way and unsubscribe. We shouldn't force it.

A mandatory EOF introduces a new way of misconfiguration that was not there before. It can't be depended on to be received. It forces me to change my existing app because I want a service to be exported. (have to add eof)

[derekcollison]

EOF is how producers signal to the requestor, has nothing to do with interest at that point.

We never assume idempotency, even with streams. e.g each stream message is its own message but it relates to a global state and order and no redelivery matter etc.

We do not want to have the customer write logic. We should help out detecting data loss etc. There are some fun ways to do this.

EOF is smarter at clean up the interest, again because we can not depend on that. Long story and maybe we can do a video chat on it.

EOF are preferred but not mandatory. And if interest is cleaned up we detect that as well now in jetstream branch of server.