ntop / nDPI

Open Source Deep Packet Inspection Software Toolkit
http://www.ntop.org
GNU Lesser General Public License v3.0
3.77k stars 892 forks source link

ndpi detect netflow/ssl anyconnect vpn only by known port ? #194

Closed Niko78 closed 8 years ago

Niko78 commented 8 years ago

hi, I use last nightly ntopng and ndpi detects strange things 1) all my routers sends netflow to ntopng on port 7002 so should be detected as netflow stream, but it is displayed as afs3-prserver in ntopng 2) we have cisco anyconnect ssl vpn access and this time it appears in ntopng as quic

I'm not saying it's not normal, I just would like to know if it's normal.

thanks

Nico

kYroL01 commented 8 years ago

Can you provide a pcap with specific captured traffic to understand better ? I don't think is normal, I think it might be a bug in dissection, but I have to check your pcap.

Thanks

Niko78 commented 8 years ago

sure first pcap about netflow stream received on ntopng server netflow.zip

Niko78 commented 8 years ago

second pcap about stream being anyconnect ssl vpn anyconnect.zip

kYroL01 commented 8 years ago

@Niko78 for point 1: I passed the pcap on nDPI and it detect perfectly as Netflow. The problem is that ntopng show only 10 pkts as Netflow, and the rest as RX or Unknown. The afs3-prserver is the keyword associated by the web to port 7002

kYroL01 commented 8 years ago

Point 2: nDPI detect correctly this pcap as SSL and OpenVPN. No errors. So try to upgrade nDPI and ntopng and recheck.

Niko78 commented 8 years ago

ah ok and what could be the solution ?

kYroL01 commented 8 years ago

I think is a things internal to ntopng and nDPI could be not well syncronized. We have to check better. I suggest first to upgrade all and try again.

Niko78 commented 8 years ago

just upgraded now I'm in ntopng 2.3.160518 1.7.1-dev-413-1e37916

Niko78 commented 8 years ago

so after the upgrade, ntopng still shows me QUIC for anyconnect ssl vpn and still afs3-prserver for netflow :-(

kYroL01 commented 8 years ago

This is what I see from anyconnect pcap on ntopng:

anyconnect

kYroL01 commented 8 years ago

and this for netflow:

filtered by Netflow application netflow_1

and with all protos enabled netflow_err

So the problem is that ntopng does not show all the Netflow pkts as netflow, but the afs3-prserver is just the name associated to port 7002, so what's the problem with this ? The coloumn Application is where the flows are detected.

And in nDPI I detect all the protos correctly.

Niko78 commented 8 years ago

ok just retried on my side

anyconnect netflow
kYroL01 commented 8 years ago

Yes, the error is on RX show for anyconnect, and this is the same for me, but i can't see wrong quic pcap... It's strange. Are u really sure to have upgrade correctly ? Because we did some modification on quic yesterday.

Niko78 commented 8 years ago

well I just upgraded ntopng/nprobe...

ntop

or I missed something else ?

kYroL01 commented 8 years ago

It's very strange. I think you didn't missing anything but I don't understand why I see correctly your pcap and you don't. I have to investigate much better.

Niko78 commented 8 years ago

if I can help you let me know

Niko78 commented 8 years ago

about netflow appearing as RX do you think a fix is possible to get netflow instead of RX ?

kYroL01 commented 8 years ago

Yes, sure, later i'll check all.

Niko78 commented 8 years ago

I think I found the issue Right now I start ntop like that ntopng -F "mysql;localhost;ntopng;flows;root;azer" -i tcp://172.31.1.244:5556 -d /storage/tmp -w 3000 -v

if I start it like that ntopng -F "mysql;localhost;ntopng;flows;root;azer" -i eth0 -d /storage/tmp -w 3000 -v it seems to be ok with this command I can even see throughput graph below whereas it doesn't work with first command

problem is that with second command I can't get all streams, I can see only packets from all my routers (netflow packets )and not trafic, no idea why

hope it will help

Nico

kYroL01 commented 8 years ago

@Niko78 It seems more a ntopng/nprobe issue than a nDPI issue, because as I explained before, with nDPI I have detected the traffic correctly.

Niko78 commented 8 years ago

ok, I posted an issue in ntopng with a link to here let's see what they think about

lucaderi commented 8 years ago

I have looked at the packets and in particular to fragments that create the issue you reported. In nDPIreader fragments are dropped, in ntopng they are handled. I have made an enhancement to nDPIreader and nDPI, so issues should be fixed. This said you have most of the traffic fragmented so IMHO you should fix your network first.