Congestion avoidance behavior

[TimonPost]

Situation

I think you know what congestion avoidance is but to recap it is shortly said an way to limit sending packets when connection is bad and when good we can send more.

Currently I am making an estimation of the round trip time (RTT). Every time we receive an acknowledgement we check the time the packet is send and we can see how long it took to between sending the packet and receiving its acknowledgement. This value we measure is the RTT. If the RTT is to high we have bad network conditions when this value is low we have good internet conditions.

So now we are at the point that if the RTT is to high we want to limit outgoing packets, however I don't think we want to limit sending packets with laminar UdpSocket. If the UdpSocket gets a packet to send it needs to send it. So that the user can know for sure that the packet pushed onto the socket will be send. So this discarding of packets when the network is bad should be an option for or needs to be under control of the end user. Imagine you queue a packet and without you knowing it will be discarded and not sent.

Of course is this pretty technical and so wee need to add some abstraction that will help the user to decide on how much packets to send based on or estimation.

Question being:

Should we control sending speed or should we make user able to control witch packets to send when network is slow. We probably need to add some abstractions etc. Please add ideas on how we can handle this congestion behavior better.

[fhaynes]

IMO we should generate an event and bubble it up to let the application handle it. Depending on the specific game, they will probably want to handle it different (DC the user, change what is being synced, notify others, etc)

[LucioFranco]

@TimonPost @fhaynes I agree, we should bubble up a network quality changed event and let the higher level application decide. If we see the need to have something like this in this library we can add it after 0.1.0.

[TimonPost]

note from @fhaynes on discord "If we start randomly dropping packets to try to compensate, then that is going to manifest as bugs or issues in the game, and the devs won't have a clue as to why if we don't notify them. We could try, for example, compressing the packets to compensate but I think anything we do should minimally intrusive or, in response to the application layer telling us to do something specific "

Oke so there would be different strategies we could use to reduce network traffic. And the user needs to decide what to do when network is bad .So we need to find ways on how to notify the user with event and how to have different strategies to accomplish network traffic.

Currently I have a descent RTT estimation with smoothing factor implemented. For now I will make sure that that works solid. And as @LucioFranco says we can have this in 0.1.0 but we can also move the abstractions and event to 0.2.0. So the basics will be in the first version.

[OvermindDL1]

For congestion avoidance the Raknet model should really be followed. Raknet uses pure UDP for note.

It has 3 style of reliability and 2 styles of ordering, of which can all be mixed into 6 variations:

• Reliable Ordered -> Receive every packet (file downloading for example) in order (any missing keeps the later ones buffered until they are received)
• Reliable Sequenced -> Toss away any packets that are older than the most recent (like a position update, you don't care about older ones), but all are received, just the application may not receive older ones if a newer one came in first.
• Reliable Unordered -> Receive every packet and immediately give to application, order does not matter.
• Unreliable Ordered -> Will be passed to the application in order with a specified waiting time to see if any older ones come in within that bound else are dropped.
• Unreliable Sequenced -> Like Reliable Sequenced but don't resend any missing ones.
• Unreliable Unordered -> Free to be dropped, used for very unnecessary data, great for 'general' position updates with an occasional reliable one.

In addition Raknet had 256 Channels and 256 Priorities, some of which were reserved for a command channel and system level priority that were unavailable to the application. Each channel had their own ordering semantics and thus could interleave freely with other channels. Priority defines what gets dropped first under times of congestion for unreliable packets. Every packet could have a different set of flags and RakNet packed it really well. Excusing a couple of bugs in the actual C++ code of RakNet, it's protocol was really solid, supporting encryption and all in addition to the above, it might be The model to copy (and it is entirely open sourced, though it really really needs to be rewritten into Rust as it just keeps having various bugs related to ownership that just wouldn't happen in Rust). It had pluggable stragies for how to buffer, how to send packets, etc... etc... It is even to this day generally used as 'The' model for optimal Game Packet handling.

[fhaynes]

RakNet is good, and certainly serves as an exemplar of networking excellence. It is only recently available as open source with the Oculus acquisition, iirc. It is also significantly more complex. Point being, it will take time and evaluation for how to fit the desired features into the engine.

[TimonPost]

We already have had some discussion about different send strategies like those you channels you described. We still need to think out about how we are going to implement this with rust. I am going to checkout Raknet to see their implementations and see if we can use things of it. Because I also saw a .net udp implementation it also had those channels you are telling about.

[OvermindDL1]

RakNet does indeed have a LOT of features that may not be necessary, including some big ones of (but not limited to):

• Network Congestion Tester -> a plugin that lets you test various forms of network congestion, random dropping, slowing, etc...
• RPC -> Really not a fan of RPC myself, though I guess some people like it...
• Automated replication of objects -> this should be built on the ECS library instead
• Voice Communication -> Could just be another library on top of this one.
• Encryption -> Well this should probably be in this library
• NAT Punchthrough -> This should probably also be in here...
• Lobby system -> This should be another library, not built in I'd think
• Difference Patcher -> Can give it a list of files or data or so and it will figure out the differences between the peers (like server to client for game files or client to server for a skin or something) and only send the parts of the files that differ instead of the whole files, this can also be a library on top of laminar but is indeed very useful.

Among a lot of other things.

I used to program on RakNet (and admin it's forums and so forth) near a decade ago back way way before it was ever free and I have no doubt I could figure out it's codebase again if you need me to go through anything of it.

For note, if you look at RakNet on the facebook github it is sorely out of date and very buggy. https://github.com/SLikeSoft/SLikeNet is a fork of it that has a lot of stability and security patches applied along with a few other useful fixups. Honestly though the code is not that useful but rather the API usage of it, the feature set, and the packet format are the parts that are very useful to emulate.