nprobe on Ubiquity, ntopng on AWS help

[rwrocket]

I feel like I am chasing my tail here. I am new to ntopng and nprobe and what I am trying to is run ntopng on AWS ubuntu 16.04 and nprobe on an Ubiquity Edgerouter.

The Edgerouter only has nprobe installed and is behind a NAT

I run this command on my nprobe Edgerouter nprobe -n none --zmq "tcp://AWSSERVERIP:1234" -i br0 --zmq-probe-mode (bridge 0 is the bridge I am passing traffic through)

01/Jun/2017 06:34:01 [nprobe.c:3368] ERROR: Invalid nProbe license (/etc/nprobe.license) [Missing license file] 01/Jun/2017 06:34:01 [nprobe.c:3375] ERROR: 01/Jun/2017 06:34:01 [nprobe.c:3376] ERROR: 01/Jun/2017 06:34:01 [nprobe.c:3377] ERROR: Switching to DEMO MODE (missing valid license) 01/Jun/2017 06:34:01 [nprobe.c:3378] ERROR: 01/Jun/2017 06:34:01 [nprobe.c:3379] ERROR: Purchase your nProbe license at 01/Jun/2017 06:34:01 [nprobe.c:3380] ERROR: https://shop.ntop.org/ 01/Jun/2017 06:34:01 [nprobe.c:3381] ERROR: 01/Jun/2017 06:34:01 [nprobe.c:3382] ERROR: 01/Jun/2017 06:34:01 [nprobe.c:7192] ERROR: 01/Jun/2017 06:34:01 [nprobe.c:7193] ERROR: NOTE: This is a DEMO version limited to 25000 flows export. 01/Jun/2017 06:34:01 [nprobe.c:7194] ERROR: nprobe: unrecognized option '--zmq-probe-modenprobe' 01/Jun/2017 06:34:01 [nprobe.c:4760] WARNING: Unrecognized option '--zmq-probe-modenprobe' 01/Jun/2017 06:34:01 [nprobe.c:4806] WARNING: The output interfaceId is set to 0: did you forget to use -Q perhaps ? 01/Jun/2017 06:34:01 [nprobe.c:4809] WARNING: The input interfaceId is set to 0: did you forget to use -u perhaps ? 01/Jun/2017 06:34:01 [nprobe.c:4909] Welcome to nProbe Pro v.7.3.160505 ($Revision: 5104 $) for x86_64-unknown-linux-gnu 01/Jun/2017 06:34:01 [nprobe.c:4919] Running on Ubuntu 14.04.4 LTS 01/Jun/2017 06:34:01 [nprobe.c:4930] [LICENSE] nProbe SystemId: 4E40BDEF499602D2 01/Jun/2017 06:34:01 [nprobe.c:7210] Welcome to nProbe v.7.3.160505 for x86_64-unknown-linux-gnu 01/Jun/2017 06:34:01 [plugin.c:1029] 0 plugin(s) enabled 01/Jun/2017 06:34:01 [nprobe.c:6766] Non IPv4/v6 traffic is discarded according to the template 01/Jun/2017 06:34:01 [nprobe.c:5419] Using packet capture length 128 01/Jun/2017 06:34:01 [nprobe.c:7383] IPv6 traffic will NOT be exported/accounted by this probe 01/Jun/2017 06:34:01 [nprobe.c:7384] due to configuration options (e.g. use NetFlow v9) 01/Jun/2017 06:34:01 [nprobe.c:7429] Flows ASs will not be computed (missing GeoIP support) 01/Jun/2017 06:34:01 [nprobe.c:7514] Capturing packets from interface br0 [snaplen: 128 bytes] 01/Jun/2017 06:34:01 [util.c:4030] Initializing ZMQ as client 01/Jun/2017 06:34:01 [util.c:4049] Exporting flows towards ZMQ endpoint tcp://AWSSERVERIP:1234 01/Jun/2017 06:34:01 [util.c:3123] nProbe changed user to 'nobody' 01/Jun/2017 06:34:01 [nprobe.c:7737] nProbe started successfully

---

ntopng side sudo ntopng /c -i "tcp://172.31.3.153:1234" --zmq-collector-mode

(I don't seem to be able to use for the IP as whenever I do it says cant create interface. so I am using the local AWS IP, if this is the problem tell me how can I run it on ? )

ubuntu@ip-172-31-3-153:~$ sudo ntopng /c -i "tcp://172.31.3.153:1234" --zmq-collector-mode ntopng: unrecognized option '--zmq-collector-mode' 01/Jun/2017 06:37:54 [Ntop.cpp:933] Setting local networks to 127.0.0.0/8 01/Jun/2017 06:37:54 [Redis.cpp:106] Successfully connected to redis 127.0.0.1:6379@0 01/Jun/2017 06:37:54 [Ntop.cpp:1152] Registered interface tcp://172.31.3.153:1234 [id: 9] 01/Jun/2017 06:37:54 [Ntop.cpp:1165] Registered interface view tcp://172.31.3.153:1234 [id: 9] 01/Jun/2017 06:37:54 [Utils.cpp:299] User changed to nobody 01/Jun/2017 06:37:54 [main.cpp:240] PID stored in file /var/tmp/ntopng.pid 01/Jun/2017 06:37:54 [HTTPserver.cpp:456] HTTPS Disabled: missing SSL certificate /usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem 01/Jun/2017 06:37:54 [HTTPserver.cpp:458] Please read https://github.com/ntop/ntopng/blob/dev/doc/README.SSL if you want to enable SSL. 01/Jun/2017 06:37:54 [HTTPserver.cpp:501] Web server dirs [/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts] 01/Jun/2017 06:37:54 [HTTPserver.cpp:504] HTTP server listening on port 3000 01/Jun/2017 06:37:54 [main.cpp:290] Working directory: /var/tmp/ntopng 01/Jun/2017 06:37:54 [main.cpp:292] Scripts/HTML pages directory: /usr/share/ntopng 01/Jun/2017 06:37:54 [Ntop.cpp:260] Welcome to ntopng x86_64 v.2.3.160415 - (C) 1998-15 ntop.org 01/Jun/2017 06:37:54 [PeriodicActivities.cpp:53] Started periodic activities loop... 01/Jun/2017 06:37:54 [RuntimePrefs.cpp:32] Dumping alerts into syslog 01/Jun/2017 06:37:54 [NetworkInterface.cpp:1426] Started packet polling on interface tcp://172.31.3.153:1234 [id: 9]... 01/Jun/2017 06:37:54 [CollectorInterface.cpp:94] Collecting flows on tcp://172.31.3.153:1234

I never seem to get any Packets

"No packet has been received yet on interface tcp://172.31.3.153:1234."

Please someone help me get this working I beg you I really want to store flows in AWS with a local mirror probe if I can work this out.

[simonemainardi]

You are using old versions of the software. Please upgrade ntopng and nProbe to the latest builds as the zmq-probe-mode.

Having the latest versions installed, you can run (note the c after the ntopng zmq endpoint port):

nprobe -n none --zmq "tcp://AWSSERVERIP:1234" -i br0 --zmq-probe-mode
sudo ntopng /c -i "tcp://172.31.3.153:1234c"

[rwrocket]

Thanks for this, I have it working now so that is a start

The traffic mbp/s seems to be inaccurate. For exmaple I run a speedtest.net through the captured network and both on the edgerouter and on the speedtest result it never goes above 20Mbps but the traffic report in ntopng is reporting a history of up to 54Mbps.

[simonemainardi]

@rwrocket it's not that data is inaccurate. The peaks you are seeing are due to the quantized nature of flows that are reported only periodically. Please see this comment that accurately explains what happens: https://github.com/ntop/ntopng/issues/958#issuecomment-277752139

Remember that, using the latest version, you can control that realtime stats refresh rate by tuning the interface page:

[emanuele-f]

Please reopen if necessary