search

ntop / nDPI

Open Source Deep Packet Inspection Software Toolkit
http://www.ntop.org
GNU Lesser General Public License v3.0
3.79k stars 892 forks source link

TLS: out-of-order fragmented Client Hello #1217

Open IvanNardi opened 3 years ago

IvanNardi commented 3 years ago

nDPI doesn't always handle out-of-order fragments of Client Hello messages in TLS sessions (see #1195 for some details about why this scenario is interesting)

The case which is not handled happens when the first received fragment is not the first fragment of the CH; all others scenarios should be fine.

Example attached: tls_frags_client_hello_out_of_order.pcapng.gz

utoni commented 3 years ago

I thought that nDPI does not support any TCP segment reassembly at all? There was a PR some time ago. But the code was somehow broken and hard to understand.

IvanNardi commented 3 years ago

I thought that nDPI does not support any TCP segment reassembly at all? There was a PR some time ago. But the code was somehow broken and hard to understand.

You are right: nDPI lacks a generic (TCP) reassembler, like the one proposed some months ago.

Nonetheless, some specific dissectors support some kind of specific "reassembly" feature: for example TLS code is able to reassemble (most of) ClientHello or certificate fragments, or Kerberos code is able to reassemble (some cases of) messages split across multiple packets.

utoni commented 3 years ago

But would it not better to write a generic TCP reassembler as other protocols may profit as well? I know that writing a reassmbler is not a trivial task and it will also increase nDPI's memory usage. But if it can increase the detection ratio it could be worth a(nother) try.

What do you think?

IvanNardi commented 3 years ago

I think that a generic reassembler code might be beneficial to nDPI: it could be used for TCP (obviously), for IP reassembler (IP fragments are rarely an issue on home/enterprise networks but they can cause troubles on networks where tunnels are involved) and even on some UDP scenarios (QUIC Client Hello or DTLS Certificate fragments). It could be also used to replace all the existing specific code I cited in the previous post.

From a development perspective, a generic reassembler (for a DPI engine, not for a network device!) shouldn't be too hard to code; you can always start handling the most common cases first and then add the hardest/uncommon cases later, in an incremental way. The real issue is the performance: with some naive implementations you end up copying the data multiple times or wasting a lot of memory.

A critical point I see is deciding when that code will be triggered. Let's take the TCP example: do we reassemble all the flows before trying to classify them, or do we reassemble only "some" flows, only in some "specific" cases? I strongly believe the former is the best option, but I know that opinions may differ on this topic. There is even the option to left TCP reassembler responsibility to the application, not to the library itself (like the flow management). The application might pass a reassembled flow to nDPI (or not) depending on external factors. A bit extreme as a solution to me, but I mention it for the sake of completeness.

utoni commented 2 years ago

1134, #1122 may be helpful for implementation