Defragmentation issue

[F5OEO]

It seems that a PDU fragmented over 2 BBFRames is not recovered and dropped. Attached screenshot to see analysis from wireshark

[F5OEO]

Debug message (from an other capture) : Seems length calculation has issue ?

2022-11-29T17:17:04Z DEBUG dvb_gse::gsepacket] extracted GSE Packet with header GSE Header (S = true, E = false, LT = broadcast label, GSE Length = 647 bytes, Fragment ID = 113, Total Length = 1348, Protocol Type = 0x0800, Label = broadcast)
[2022-11-29T17:17:04Z DEBUG dvb_gse::gsepacket] start of GSE fragment ID = 113
[2022-11-29T17:17:04Z DEBUG dvb_gse::gsepacket] no more GSE Packets in BBFRAME
[2022-11-29T17:17:04Z DEBUG dvb_gse::gsepacket] extracted GSE Packet with header GSE Header (S = false, E = true, LT = broadcast label, GSE Length = 707 bytes, Fragment ID = 113)
[2022-11-29T17:17:04Z DEBUG dvb_gse::gsepacket] pushing non-start GSE fragment ID = 113
[2022-11-29T17:17:04Z DEBUG dvb_gse::gsepacket] end of GSE fragment ID = 113
[2022-11-29T17:17:04Z DEBUG dvb_gse::gsepacket] defragmented length 1346 does not match total length 1348

[daniestevez]

Hi, I have tested defragmentation by transmitting large ICMP packets with gr-dvbgse, so I'm surprised that this fails.

Let's check the sizes. The first fragment has a GSE length of 647 bytes. The header has a fragment ID field (1 byte), a total length field (2 bytes) and a protocol type field (2 bytes), for a total of 5 bytes. This means that the payload is 642 bytes. The second fragment has a GSE length of 707 bytes. Its header has a fragment ID field only (1 byte). However, since this packet is the last, it has a CRC-32 field at the end, whose size is counted in the GSE length. Therefore, besides payload, we have 5 extra bytes being counted in the GSE length. The payload is then 702 bytes.

In total, we have 1344 ( = 642 + 702) bytes of payload. The size of the protocol type field (2 bytes) is also counted towards the total length field. So the value of the total length field should be 1346. However, it is 1348, and this is what causes the packet to be dropped.

My understanding is that the total length field in the first packet does not have the correct value. Do my calculations make sense? Do you know the expected size of the payload, so that we can check that the problem is that the total length is too large, rather than some of the fragments being too short?

[daniestevez]

The relevant information from TS 102 606-1 is as follows:

GSE_Length: This 12-bit field indicates the number of bytes following in this GSE Packet, counted from the byte following this GSE Length field. The GSE Length field allows for a length of up to 4 096 bytes for a GSE Packet. The GSE Length field points to the start of the following GSE Packet. If the GSE packet is the last in the Base Band frame it points to the end of the Base Band frame Data Field or the start of the padding field. For End packets, it also covers the CRC_32 field.

Total_Length: This field is present in the GSE Header carrying the first fragment of a fragmented PDU. The 16-bit field carries the value of the total length, defined as the length, in bytes, of the Protocol Type, Label (6 byte Label or 3 byte Label), Extension Headers, and the full PDU. The receiver shall perform a total length check after reassembly. It may also use the total length information for pre-allocation of buffer space. Although the length of a single GSE Packet is limited to almost 4 096 bytes, larger PDUs are supported through fragmentation, up to a total length of 65 536 bytes.

NOTE 2: Since the information in the total length field is intended for use by higher layers in the reassembly process, the length of the CRC_32 field is therefore not included in the Total_Length.

[F5OEO]

The total length field shoud be 1346, I guess...See a gse packet end and start well handled.

[daniestevez]

This packet has GSE Length field 1346. It has a protocol type field (2 bytes). Therefore, the PDU size (the size of the IPv4 payload) should be 1344 bytes. If the fragmented packet that you showed earlier has the same IPv4 payload size, then I think that my calculations check up (I got a payload size of 1344 bytes also) and the total length field produced by this transmitter is not correct, since I don't see how with 1344 bytes of payload one could get a total length of 1348, given that the CRC-32 doesn't count and there is no label or extension headers, so we can only add 2 bytes for the protocol type.

[daniestevez]

If this is a transmitter that you can't fix, you could try commenting out L214-L248 in gsepacket.rs. If this works (there is also the CRC check that needs to be passed to consider the defragmented packet as valid), then we might have an option to disable the total length field check.

However, if we're right that the total length field in this transmitter is wrong, it would be very advisable to fix the transmitter, since other receivers could have the same problem.

[F5OEO]

Commenting this is and get a stream much better..and nearly no error stream. Transmitter is from Newtec EL170, a reference in the dvb field , usually implementing new features prior to standards. This is a second hand equipment, and could not access to firmware update (if available). Your analysis seems OK : the only explanation, for a 2 bytes mismatch could be the protocol type counted on the last frag (not present) ?

[F5OEO]

Here is a pcap capture from Longmynd BBframes. Hope you can replay it. My dvb-gse command : dvb-gse --listen 230.0.0.1:1234 --tun tun0 --isi 171 gsebbframe.pcapng.tar.gz

[F5OEO]

This packet has GSE Length field 1346. It has a protocol type field (2 bytes). Therefore, the PDU size (the size of the IPv4 payload) should be 1344 bytes. If the fragmented packet that you showed earlier has the same IPv4 payload size, then I think that my calculations check up (I got a payload size of 1344 bytes also) and the total length field produced by this transmitter is not correct, since I don't see how with 1344 bytes of payload one could get a total length of 1348, given that the CRC-32 doesn't count and there is no label or extension headers, so we can only add 2 bytes for the protocol type.

And I agree with this calculation....weird !

[F5OEO]

By using an other defrag tool, same issue : length mismatch (2 bytes). An other issue discovered, not standard compliant (but not as important : label type should be 11 at end of frame).

In conclusion : this seems to be the transmitter not standard compliant. Suggestion (as it could be encountered on other transponders) : treat it as a warning, but not as critical (maybe with an option --permissif-standard)

[daniestevez]

Hi. My plan is to add an option so that this check can be disabled (turning it into a warning sounds good until we realize that there are potentially many of these packets per second, which would spam the log). This is a simple change, so I'll probably do it this week.

Indeed you're right about the label. In the second fragment it is set to 10 rather than 11. This is not something that my dvb-gse implementation cares about. I think it only looks at the label of the first fragment, and then considers that to be the value of the label for the defragmented PDU.

[daniestevez]

I have added a pull request #12 that adds an option to skip the checking of the total length field. I haven't tested this, but in principle it should solve the problems you have with the Newtec EL170. Can you give it a try?

By the way, no need to close this issue manually. It will get closed automatically when the pull request is merged.

[F5OEO]

Confirm that it fixes the issue. Without skip-total-length, lot of bbframes skipped, with option, all is OK.

[daniestevez]

Thanks for testing. Merged.