PF_RING FT using user defined protocol file not work

[WangGaofei]

Dear PF_RING FT developers,

We are using latest PF_RING FT with built-in nDPI integration. We want to add some custom subprotocols using protocol.conf file.

OS: Centos 8 and Ubuntu 22.04

Only we find

1. Port rule works
2. host rule doesn't work

cat ./proto_youtube.txt host:"youtube.com"@MyYouTube // not work host:"www.youtube.com"@MyYouTube // not work host:"*.youtube.com"@MyYouTube // not work

When I pass protocol file, nothing changes. However, when I use ndpi_reader with protocol file directly, it works.

./ftflow_pcap -i ./youtube.pcap -7 -v 2 [Flow] l7: TLS.YouTube, category: 1, tunnelType: 0, srcIp: 192.168.5.131, dstIp: 64.233.170.91, srcPort: 51836, dstPort: 443, protocol: 6, tcpFlags: 0x1A, c2s: { Packets: 15, Bytes: 3286, First: 1706165914.157578, Last: 1706165963.745968 }, s2c: { Packets: 19, Bytes: 14373, First: 1706165914.169150, Last: 1706165964.65586 }

./ftflow_pcap -i ./youtube.pcap -7 -v 2 -p ./proto_youtube.txt [Flow] l7: TLS.YouTube, category: 1, tunnelType: 0, srcIp: 192.168.5.131, dstIp: 64.233.170.91, srcPort: 51836, dstPort: 443, protocol: 6, tcpFlags: 0x1A, c2s: { Packets: 15, Bytes: 3286, First: 1706165914.157578, Last: 1706165963.745968 }, s2c: { Packets: 19, Bytes: 14373, First: 1706165914.169150, Last: 1706165964.65586 }

Can you give use some ideas, thank you.

[cardigliano]

Please specify the PF_RING FT and nDPI version

[swtar]

你好，邮件已收到！

[WangGaofei]

Please specify the PF_RING FT and nDPI version

PF_RING FT 8.6.1.23102 nDPI 4.8.0-4331-816b001

[WangGaofei]

Please specify the PF_RING FT and nDPI version

Can you help us to solve this issue?

PF_RING FT 8.6.1.23102 nDPI 4.8.0-4331-816b001

Maybe the PF_RING FT framework misses some configuration or nDPI API.

[cardigliano]

Checking this..

[WangGaofei]

Checking this..

Thank you. By the way, It doesn't support regex. In the previous ndpi version, hyperscan is removed. And there is a pcre compile option, but it doesn't work too.

[cardigliano]

@WangGaofei could you provide the youtube.pcap pcap to run the same test?

[WangGaofei]

@WangGaofei could you provide the youtube.pcap pcap to run the same test?

Yes, we have tested the same youtube.pcap against nDPI. We found that nDPI only works with finxed string, not for regex and wildcard.

Please see the line 25 in https://github.com/ntop/nDPI/blob/dev/example/protos.txt Line 25-> host:"*.lvlt.dash.us.aiv-cdn.net.c.footprint.net"@AmazonVideo I am not sure whether nDPI suppports regex or wildcard or not.

In our case, we have to make some subprotocol based this rule. media-*.whatsapp.net -> WhatsAppUpload etc.

[cardigliano]

@WangGaofei nDPI supports wildcards at the beginning of the string, not full regex (for performance reasons)

[WangGaofei]

@WangGaofei nDPI supports wildcards at the beginning of the string, not full regex (for performance reasons)

@cardigliano Thank you. Understood.

In my previous project, there is a similar requirement. I do the following steps.

1. Split the wildcard into tokens by *, and find the longest or uniq one token, mark the token and the rule pair
2. Use corasick to fast search matched token, then get the relevant rules, and then test and find first matched rule
3. Return the result

I have another question. Why nDPI droped hyperscan implementation? It's a great tool.

[cardigliano]

This is a question for the nDPI community :-) https://github.com/ntop/nDPI/issues

[WangGaofei]

This is a question for the nDPI community :-) https://github.com/ntop/nDPI/issues

Thank you :-)