1 Megabyte `SendType::Unreliable` Unreliability

[ActuallyHappening]

Thanks for this awesome crate!

I have encountered an issue with renet which first surfaced here: https://github.com/lifescapegame/bevy_replicon/issues/107 In a SendType::Unreliable renet channel, I can't reliably send payloads of hundreds of kilobytes:

From tests: 20 311: Sends consistently 40 693: Sends consistently 73 083: Never sends 329 359: Never sends

If this is intentional, can it be documented somewhere? I don't want this to be the case, however. I'm not sure if 'packet fragmentation' is the usual solution to this sort of issue, and would appreciate pointers on how to sync large amount of data through Unreliable channels, well, reliably.

[lucaspoffo]

Unreliable packets above a certain size are very susceptible to packet loss. Unreliable packets above 1200 bytes are divided in fragments of 1200 bytes, and those are sent unreliable. If any fragment is lost, the whole message is lost (that's why large packets sent unreliable are lost easily).

We could warn the user when a large message is sent unreliable and suggest sending it reliably (also adding something to the README).

For example, in the SteamNetworkSocket, if a packet has more than 15 fragments, they warn the user about it, and it's converted to reliable automatically: https://github.com/ValveSoftware/GameNetworkingSockets/blob/de03d74226eb3b9a299e05f5fff93965d3dce2d9/src/steamnetworkingsockets/clientlib/steamnetworkingsockets_snp.h#L55C34-L55C34 https://github.com/ValveSoftware/GameNetworkingSockets/blob/de03d74226eb3b9a299e05f5fff93965d3dce2d9/src/steamnetworkingsockets/clientlib/steamnetworkingsockets_snp.cpp#L325C13-L325C13

[ActuallyHappening]

Yes adding a warning, whether a trace::warn!("Large packets sent unreliably are susceptible to packet loss, consider breaking your {} byte packet into smaller chunks or sending reliably withSendType::Reliable") or a trace::debug!() would help greatly! Then people in my position would need only to look at their logs instead of spending 4 days not knowing what is causing this issue. Maybe some documentation on the SendType::Unreliable enum variant would help as well.

When you say

unreliable packets above 1200 bytes are divided in fragments of 1200 bytes

is that done automatically by renet? I presume it is, that's what 'packet fragmentation' means right?

[lucaspoffo]

is that done automatically by renet? I presume it is, that's what 'packet fragmentation' means right?

Yes, it detects automatically. Messages below 1200 bytes are grouped together as many can fit in the 1200 packet. Packets above it are sliced in fragments and sent separately. This works the same for reliable/unreliable, the difference is with reliable we track fragments/messages lost and resend them.

[Shatur]

Is it expected to have any loss even on local machine? Also Is it expected that client never receives any message if you constantly sending it? Because in bevy_replicon we send the replication data over unreliable channel since the last ack from client and looks like in the author's case client doesn't receive any message: https://github.com/lifescapegame/bevy_replicon/issues/107

[ActuallyHappening]

Yeah, the client doesn't seem to receive any payload larger than about 40 kilobytes even on my local machine. To fix this, I would @Shatur send messages over a (configurable) limit to send with SendType::Reliable. That seems like the quickest, easiest and simplest solution.

[Shatur]

To fix this, I would @Shatur send messages over a (configurable) limit to send with SendType::Reliable

It's a terrible idea to use reliable channel for replication data. You don't need any previous message if you received the latest one. Also could you confirm that the echo example works with reliable channel when you send 40 kilobytes?

[ActuallyHappening]

Yes the echo example works like a charm over Reliable I haven't tested in bevy_relicon going over reliable because that would involve patching in a few things

[ActuallyHappening]

To fix this, I would @Shatur send messages over a (configurable) limit to send with SendType::Reliable

It's a terrible idea to use reliable channel for replication data. You don't need any previous message if you received the latest one.

Well not always, the components I was syncing when I hit this issue made up the initial state of my world. Since I have to send this state, preferably only once, not sending it reliably doesn't guarantee that it makes. I agree 99% of the data transferred after this would be slowed down if not sent unreliably. For this use case, maybe a bevy_replicon specific api could be added, .replicate_reliably, that conditionally sends reliably?

[Shatur]

Then you probably want to send the state via event (you can configure reliability for it). But it's better to ask such questions in bevy_replicon issues or in Bevy discord in dedicated thread.

I also suspect that something is wrong with unreliable channel here because the message never gets received.

[lucaspoffo]

You can enable the trace log for renet to check better what happens with the sliced packets. Every fragment that is received will log a message like: Received slice 10 from message 123. (20/40). Also, there are some warnings of dropped and other stuff that you can check.

[lucaspoffo]

Added the warn in 0aa1a4d08fde68a04df923ff553a0050801bef15