is there any relation in coAP and BT?

[JeevanNailwal]

Hi All, I am looking to avoid BT data in my network. I was testing latest DEV branch and found the below result:

Detected protocols: Unknown packets: 161 bytes: 16355 flows: 64
DNS packets: 19 bytes: 1793 flows: 9
HTTP packets: 75 bytes: 16067 flows: 6
SSDP packets: 2 bytes: 540 flows: 2
COAP packets: 7994 bytes: 5861897 flows: 268
BitTorrent packets: 191 bytes: 29158 flows: 68
ICMP packets: 6 bytes: 510 flows: 5
SSL packets: 65 bytes: 18640 flows: 3
HTTP_Proxy packets: 3 bytes: 202 flows: 1

I had uploaded the relevent pcap file at: https://drive.google.com/open?id=0B4M_moAIGLMSZUVFN0cwUF8yMnc

I am not sure if this recently aadded CoAP protocol and BT are linked, Can some one please provide information on same? for my experiment, Do I need to avoid CoAP packets too? Will it impact my other services?

Regards, Jeevan

[kYroL01]

There isn't direct relation between coap and BT. As explain here, COAP "is an application layer protocol that is intended for use in resource-constrained internet devices, such as WSN nodes. CoAP is designed to easily translate to HTTP for simplified integration with the web, while also meeting specialized requirements such as multicast support, very low overhead, and simplicity".

I need to see your pcap (please accept my request to download the pcap) to understand better.

In your session you captured, the main application is BT ? Try to provide a pcap with only a BT session.

[JeevanNailwal]

Hi Michele, The provided pcap has only session with BT. the o/p of ndpiReader output was pasted in my query. I am pasting the same again, for your reference.

Detected protocols: Unknown packets: 161 bytes: 16355 flows: 64

DNS packets: 19 bytes: 1793 flows: 9 HTTP packets: 75 bytes: 16067 flows: 6 SSDP packets: 2 bytes: 540 flows: 2 COAP packets: 7994 bytes: 5861897 flows: 268 BitTorrent packets: 191 bytes: 29158 flows: 68 ICMP packets: 6 bytes: 510 flows: 5 SSL packets: 65 bytes: 18640 flows: 3 HTTP_Proxy packets: 3 bytes: 202 flows: 1

I had accepted your request for google drive ( and made it public so that everyone can view it now) More Info: I am using Transmission bit-torrent client to start this session and downloaded around 6 MB data.

Confusion with DPI output: As I said I want to block bit torrent data in my network, But my problem is that DPI is detetcting 90% packets as CoAP and seems only initial transactions(where peer/trackers information gets shared) are getting counted as bittorrent.

Regards, Jeevan

On Sat, Mar 26, 2016 at 7:19 PM, Michele Campus notifications@github.com wrote:

There isn't direct relation between coap and BT. As explain here, COAP https://en.wikipedia.org/wiki/Constrained_Application_Protocol "is an application layer protocol that is intended for use in resource-constrained internet devices, such as WSN nodes. CoAP is designed to easily translate to HTTP for simplified integration with the web, while also meeting specialized requirements such as multicast support, very low overhead, and simplicity".

I need to see your pcap (please accept my request to download the pcap) to understand better.

In your session you captured, the main application is BT ? Try to provide a pcap with only a BT session.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/ntop/nDPI/issues/164#issuecomment-201826250

[kYroL01]

Thanks @JeevanNailwal for the pcap. I think you are right! Definetly that are not CoAP prackets but BT !!

I need to fix this! Thanks. Hope to solve asap !

[kYroL01]

Now I think you could be satisfy. Result from your pcap after fix:

Unknown packets: 146 bytes: 13776 flows: 62
DNS packets: 19 bytes: 1793 flows: 9
HTTP packets: 73 bytes: 15935 flows: 6
SSDP packets: 2 bytes: 540 flows: 2
COAP packets: 15 bytes: 1200 flows: 15
BitTorrent packets: 8134 bytes: 5880059 flows: 307
ICMP packets: 6 bytes: 510 flows: 5
SSL packets: 57 bytes: 18013 flows: 3
HTTP_Proxy packets: 3 bytes: 202 flows: 1

Thanks to this issue I understand also that is necessary a big refactor of coap dissector.

[JeevanNailwal]

Hi all, But with live packets, Theres a NEW problem: (please have a look at number of UNKOWN packets)

Detected protocols: Unknown packets: 3812 bytes: 4577759 flows: 172
DNS packets: 6 bytes: 628 flows: 3
COAP packets: 3 bytes: 216 flows: 2
BitTorrent packets: 6478 bytes: 5264271 flows: 261
ICMP packets: 1 bytes: 162 flows: 1
SSL packets: 6 bytes: 446 flows: 2
Tor packets: 535 bytes: 568540 flows: 2

PCAP file for same is attached at: https://drive.google.com/file/d/0B4M_moAIGLMSNklhb2dyUGRhV1U/view?usp=sharing

[JeevanNailwal]

Hi All, There's a new issue with the above changes. Now my unknown flows are also increasing with the same rate. Please see the o/p of ndpiReader below:

Detected protocols: Unknown packets: 3812 bytes: 4577759 flows: 172

DNS packets: 6 bytes: 628 flows: 3

COAP packets: 3 bytes: 216 flows: 2

BitTorrent packets: 6478 bytes: 5264271 flows: 261

ICMP packets: 1 bytes: 162 flows: 1

SSL packets: 6 bytes: 446 flows: 2

Tor packets: 535 bytes: 568540 flows: 2

PCAP file for same is attached at: https://drive.google.com/file/d/0B4M_moAIGLMSNklhb2dyUGRhV1U/view?usp=sharing

Regards, Jeevan

On Sun, Mar 27, 2016 at 6:14 AM, Michele Campus notifications@github.com wrote:

Now I think you could be satisfy. Result from your pcap after fix:

Unknown packets: 146 bytes: 13776 flows: 62

DNS packets: 19 bytes: 1793 flows: 9

HTTP packets: 73 bytes: 15935 flows: 6

SSDP packets: 2 bytes: 540 flows: 2

COAP packets: 15 bytes: 1200 flows: 15

BitTorrent packets: 8134 bytes: 5880059 flows: 307

ICMP packets: 6 bytes: 510 flows: 5

SSL packets: 57 bytes: 18013 flows: 3

HTTP_Proxy packets: 3 bytes: 202 flows: 1

Thanks to this issue I understand also that is necessary a big refactor of coap dissector.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/ntop/nDPI/issues/164#issuecomment-201964444

[kYroL01]

I see. I'll check where is the problem.

[kYroL01]

TOR and BitTorrent need to be checked better. I ask you just please to be patient. I try to understand better in these days. Thanks.

[lucaderi]

@kYroL01 Hi Michele can we meet to discuss about how to address this issue?

[JeevanNailwal]

Thanks for the concern Michele. I will patiently wait for your response.

Regards, Jeevan

On Tue, Mar 29, 2016 at 5:11 AM, Luca Deri notifications@github.com wrote:

@kYroL01 https://github.com/kYroL01 Hi Michele can we meet to discuss about how to address this issue?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/ntop/nDPI/issues/164#issuecomment-202627808

[JeevanNailwal]

Another observation: I tried to print the return type of "ndpi_detection_process_packet" and figured that for the packet, which are not getting detected as BITTORRENT, master_protocol and protocol both are zero! While for the detected packets, protocol = 37(BITTORRENT) while master_protocol is always zero!

[kYroL01]

@lucaderi When you want.

@JeevanNailwal when a packet is unknow is normal that you have master_protocol and protocol to zero; is also normal that not all the protocols have master_protocol != 0. For example if BT is detected on SSL connection (i.e. through certificate) you have master_protocol = SSL and protocol = BT. If BT is detected directly with its dissector, you have just protocol. This is a choice as I explain you in issue #163 .

[kYroL01]

@JeevanNailwal To solve the thing that you said we have to refact the bittorrent dissector, because we need to keep flows in a cache to understand when many flows must be recognize as BT. Not a fast refactor so please be patient for the moment. Thank you!