search

NEAT-project / neat

A New, Evolutive API and Transport-Layer Architecture for the Internet
https://www.neat-project.org/
BSD 3-Clause "New" or "Revised" License
66 stars 19 forks source link

Lifetime of a flow - closing procedure #316

Closed weinrank closed 7 years ago

weinrank commented 7 years ago

Hey all,

imho the documentation is not very clear when it comes to the closing procedure of a flow. And the examples aren't either... 🤕

Goal for this topic: An understandable and documented way how the closing procedure works. My understanding how this should/can work.

img_8046

neat_shutdown

neat_shutdown initiates a graceful connection shutdown, indicating: "I have nothing more to say" while still listening to the flow and interested in events.

neat_close

neat_close closes the the flow, the application is not interested anymore. The last call. All application associated data (e.g. opCB->userData) should be released before calling this function

on_close

Comments/improvements welcome

mwelzl commented 7 years ago

Hi,

I’m not coding NEAT myself, so my apologies for mistakes. As you know (we talked about this in the past), I generally agree with this - but I have two minor comments below. I also wonder, will there be a similar discussion for opening?

(is it clear to everyone that opening a flow may not by itself cause an “accept”-like event on the other side?) Because this arrives as email in a strange and scary alien email format, possibly HTML, I’ll mark my answers with [MW] just in case…

On Apr 26, 2017, at 7:05 PM, Felix Weinrank notifications@github.com wrote:

Hey all,

imho the documentation is not very clear when it comes to the closing procedure of a flow. And the examples aren't either... 🤕

Goal for this topic: An understandable and documented way how the closing procedure works. My understanding how this should/can work.

https://cloud.githubusercontent.com/assets/6726222/25444499/ad38914c-2aab-11e7-9ad7-97a4c2d62037.jpg neat_shutdown

neat_shutdown initiates a graceful connection shutdown, indicating: "I have nothing more to say" while still listening to the flow and interested in events.

[MW] Actually I think that the phrase that you keep using about this, “I have nothing more to say” should be changed, because I think it creates a confusion:

Saying “I have nothing more to say” indicates that these are the ONLY underlying semantics - but then, if both sides send “I have nothing more to say”, you have TCP’s shutdown.

I therefore think you should talk about this as “I have nothing more to say, and I might not even read anymore”. This difference doesn’t matter very much to the application issuing it, but you have then created a description of the meaning of “shutdown” that also match the semantics of the message that would go on the wire, and what it means to the peer receiving it. (It’s important for the peer to know that this message indicates that the other side may not even be interested in receiving data anymore, as this is where we differ from TCP.)

all data written to the neat layer ist transmitted (outgoing flow buffer is drained) neat_write can't be called anymore, on_writable is not fired anymore on_all_written will still be called when if the flow was draining if the neat_layer has data in the receive buffer, on_readable will still be called and this data can still be read by neat_read new data may be read from the network socket and announced to the application - depending on the protocol neat_close

neat_close closes the the flow, the application is not interested anymore. The last call. All application associated data (e.g. opCB->userData) should be released before calling this function

connection will be closed gracefully (if not already done) all data written to the neat layer ist transmitted (outgoing flow buffer is drained) all data in the receive buffer is discarded neat_write and neat_read can not be called anymore no callback is fired anymore neat will free all flow related resources for the user this function must be called by the application on_close

indicates that the flow has successfully been closed, no on_readable or on_writable callbacks will be fired anymore and neat_read / neat_write will fail Comments/improvements welcome

[MW] I wonder if we also need an “abort” that does not require to draing the outgoing flow buffer? Don't we have that? Maybe you just didn’t talk about it because it’s the simpler case…

Cheers, Michael

adventureloop commented 7 years ago

(is it clear to everyone that opening a flow may not by itself cause an “accept”-like event on the other side?)

This isn't clear to me, how would a recv side NEAT app get the "accepted" flow with a on_connected event being triggered?

mwelzl commented 7 years ago

On Apr 27, 2017, at 12:23 PM, Tom Jones notifications@github.com wrote:

(is it clear to everyone that opening a flow may not by itself cause an “accept”-like event on the other side?)

This isn't clear to me, how would a recv side NEAT app get the "accepted" flow with a on_connected event being triggered?

I think on_connected would be triggered anyhow, but it might only happen when the sender sends the first piece of data, not necessarily as a reaction to neat_open. So, the sequence can be: Sender: open; nothing happens then, sender: send => this produces a piece of data on the wire, and when that arrives, it’s the first indication to the peer that this new NEAT flow exists, triggering on_connected and on_readable. (I hope I say this correctly)

This is necessary to map flows to streams. We discussed this in Paris; see slide 16 here: https://secure.mlab.no/neatwiki/lib/exe/fetch.php?media=slides:paris-jan2017:neat-paris2017-destreams_flowlifetime.pptx https://secure.mlab.no/neatwiki/lib/exe/fetch.php?media=slides:paris-jan2017:neat-paris2017-destreams_flowlifetime.pptx

I hope this helps…

Cheers, Michael

adventureloop commented 7 years ago

Okay, this is what I thought you might be getting at. This is the case with UDP where on_connected returns immediately.

mwelzl commented 7 years ago

Ahhh, UDP too, of course!

On Apr 28, 2017, at 10:43 AM, Tom Jones notifications@github.com wrote:

Okay, this is what I thought you might be getting at. This is the case with UDP where on_connected returns immediately.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NEAT-project/neat/issues/316#issuecomment-297942992, or mute the thread https://github.com/notifications/unsubscribe-auth/AGwOrLCRz1rjT6vSOPo0NHev0mhWGp-wks5r0abJgaJpZM4NJJ_N.

weinrank commented 7 years ago

We've had an extensive discussion on this topic today, intermediate results:

neat_close

on_close

neat_shutdown, neat_abort

Comments?

mwelzl commented 7 years ago

This seems to me to reflect an intermediate state of discussion: the agreement we had as we left the restaurant, but not what we started talking about afterwards. Let’s talk a little more tonight.

In short: I don’t think shutdown can be removed - but let’s try to talk this over before sending many emails about it.

On May 4, 2017, at 4:08 PM, Felix Weinrank notifications@github.com wrote:

We've had an extensive discussion on this topic today, intermediate results:

neat_close

closes the the flow, the application is not able to send or receive data via neat_send or neat_receive after calling this function no transmission related callbacks like on_readable, on_writable or on_all_written are called afterwards outstanding data in the NEAT's receive and send buffer gets discarded when finished, the on_close callback is fired on_close

the flow has been closed, either by actively calling neat_close or because the remote side has closed the underlying connection no transmission related callbacks like on_readable, on_writable, on_close or on_all_written are called afterwards the application should free all flow related data (e.g. userData) the application may safely terminate the event loop neat_shutdown, neat_abort

both functions get removed from the API Comments?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NEAT-project/neat/issues/316#issuecomment-299195601, or mute the thread https://github.com/notifications/unsubscribe-auth/AGwOrJsuPoNHl2TKf-2E3kYWiYzdS0oLks5r2dvGgaJpZM4NJJ_N.

mwelzl commented 7 years ago

Dear all,

Here’s an update of the update, after the discussion Felix and I just had. This is to ensure that data can be reliably transferred with the connection closing right after it. I’m marking the updates with [MW] inline (this is the part where you folks understand that I’m typing this from a mail client and not using github ;-) )

On May 4, 2017, at 4:08 PM, Felix Weinrank notifications@github.com wrote:

We've had an extensive discussion on this topic today, intermediate results:

neat_close

closes the the flow, the application is not able to send or receive data via neat_send or neat_receive after calling this function no transmission related callbacks like on_readable, on_writable or on_all_written are called afterwards [MW] outstanding data in the NEAT's receive buffer gets discarded. NEAT will try to transfer any outstanding data in its send buffer; applications that want to ensure reliable transmission of such data must rely on information provided via on_close. when finished, the on_close callback is fired on_close

the flow has been closed, either by actively calling neat_close or because the remote side has closed the underlying connection no transmission related callbacks like on_readable, on_writable, on_close or on_all_written are called afterwards the application should free all flow related data (e.g. userData) the application may safely terminate the event loop

  • [MW] • if outstanding data in the NEAT's receive and/or send buffer gets discarded, the application gets informed via a flag / notification

neat_shutdown, neat_abort

both functions get removed from the API [MW] Just for the record, I agree.