xtaci / kcp-go

A Crypto-Secure Reliable-UDP Library for golang with FEC
MIT License
4.05k stars 730 forks source link

Data splitting and merging #260

Open 89trillion-wuyanqing opened 5 months ago

89trillion-wuyanqing commented 5 months ago

I have implemented the server and client using the kcp-go framework. The server sent a data packet larger than 1500 bytes. I expected the server to split this packet, and then the client would merge it upon return. However, the result was that the server split the packet into multiple packets, and the client ended up receiving the fragmented packets, rendering them unusable.

In your WriteBuffers method, you first split the sent data according to the maximum transmission unit parameter (mss) and pass the segmented byte data to the Send method. Then, in the Send method, the logic of splitting the data into fragments is applied. As a result, the data packet will never exceed the maximum transmission unit, and therefore, the logic of splitting and merging will never be triggered.

I believe there needs to be a modification in the framework. At the user level, after sending a data packet, the receiving end should receive a complete packet instead of it being split into multiple fragments.

Therefore, I suggest that the WriteBuffers method does not need to split the sent data according to the maximum transmission unit parameter (mss). Instead, the original data should be passed directly to the Send method.

I'm not a native English speaker, so my expression may not be very accurate. Please understand.

Snipaste_2024-03-29_23-27-53
xtaci commented 5 months ago

You can encode like this: |size|payload| , try to add a size prefix before the payload.

89trillion-wuyanqing commented 3 months ago

I don't quite understand what you mean, can you tell in which method this parameter should be set.

xtaci commented 3 months ago

why not just add a prefix before the data, the prefix suggests how many bytes are there in the packet. |size|payload|size|payload|.... |size|payload|

89trillion-wuyanqing commented 3 months ago

Do you mean that before calling the Write method, I split the binary data myself, and each split data packet needs to add a 2-byte prefix to indicate the size of the data packet?

Snipaste_2024-05-20_22-01-23

In fact, my core question is whether the work of splitting data packets is implemented in the third-party package or by the user himself?

xtaci commented 3 months ago

users should be in charge of managing the border of packets. In general, kcp-go provides a realiable stream only.

89trillion-wuyanqing commented 3 months ago

When I looked at the code, I found that you would split the large data packets into segments, so I thought that the third-party package had already encapsulated the data splitting work.

Snipaste_2024-05-21_21-56-49

Because after I modified this code, when sending a large data packet, the simulated client can normally receive a complete assembled data packet, so I think that this data splitting work needs to be encapsulated, and the user is not responsible for managing the boundaries of the data packet.

Snipaste_2024-03-29_23-27-53

So if I make the above changes, will there be any serious consequences?

xfnyl commented 1 month ago

I encountered the same problem as you, and my solution is the same as yours, which is to modify the WriteBuffers() method so that it does not split packets before calling kcp.Send(). I think if streaming data is sent, the client will automatically combine the packets, and it will not matter whether it is changed or not. But if it is a single large packet that exceeds the MSS, the client will not automatically combine the packets, so this processing method is needed.