(code=killed, signal=KILL)

[MaoPann]

Now I want to use n2disk to export the stream to ntopng for monitoring, but starting n2disk fails and reports the following error： n2disk@test.service - n2disk ultra-high-speed traffic recorder with realtime indexing on test Loaded: loaded (/etc/systemd/system/n2disk@.service; disabled; vendor preset: enabled) Active: activating (auto-restart) (Result: signal) since Wed 2021-07-21 14:35:36 +08; 1s ago Process: 3234 ExecStopPost=/bin/sh -c /bin/echo "$(/bin/date) n2disk test StopPost" >> /var/log/ntop-systemd.log (code=exited, status=0/SUCCESS) Process: 3233 ExecStopPost=/bin/rm -rf /run/n2disk-test.conf (code=exited, status=0/SUCCESS) Process: 3232 ExecStopPost=/bin/rm -rf /run/n2disk-test.env (code=exited, status=0/SUCCESS) Process: 3215 ExecStartPost=/bin/sh -c /bin/echo "$(/bin/date) n2disk test StartPost" >> /var/log/ntop-systemd.log (code=exited, status=0/SUCCESS) Process: 3214 ExecStart=/usr/bin/stdbuf -oL /usr/bin/${N2DISK_BINARY} /run/n2disk-test.conf (code=killed, signal=KILL) Process: 3212 ExecStartPre=/bin/sh -c /bin/sed "/-P.$|--daemon.|--pid.*/s/^/#/" /etc/n2disk/n2disk-test.conf > /run/n2disk-test.conf (code=exited, st Process: 3178 ExecStartPre=/bin/sh -c /usr/bin/n2disk --check-license | /bin/grep "Ok|Time-Limited" && /bin/echo "N2DISK_BINARY=n2disk" > /run/n2disk-te Process: 3120 ExecStartPre=/bin/sh -c /usr/bin/n2disk5g --check-license | /bin/grep "Ok|Time-Limited" && /bin/echo "N2DISK_BINARY=n2disk5g" > /run/n2dis Process: 3111 ExecStartPre=/bin/sh -c /usr/bin/n2disk1g --check-license | /bin/grep "Ok|Time-Limited" && /bin/echo "N2DISK_BINARY=n2disk1g" > /run/n2dis Process: 3109 ExecStartPre=/bin/sh -c /bin/echo "N2DISK_BINARY=n2disk" > /run/n2disk-test.env (code=exited, status=0/SUCCESS) Process: 3106 ExecStartPre=/bin/sh -c /bin/echo "$(/bin/date) n2disk test StartPre" >> /var/log/ntop-systemd.log (code=exited, status=0/SUCCESS) Main PID: 3214 (code=killed, signal=KILL)

n2disk.conf

--interface=ens33 --dump-directory=/storage/n2disk/pcap --timeline-dir=/storage/n2disk/timeline --disk-limit=512 # --max-file-len=1000 --buffer-len=4000 --max-file-duration=60 --index --snaplen=1536 # --writer-cpu-affinity=0 --reader-cpu-affinity=1 --compressor-cpu-affinity=2,3 --index-on-compressor-threads # -u=ntopng --zmq=tcp://127.0.0.1:5556 --zmq-probe-mode --zmq-export-flows

ntopng.conf

-i tcp://127.0.0.2:5556 -w=3001 -F=nindex -m="192.168.0.0/24,192.168.1.0/24" -G=/var/run/ntopng.pid

[cardigliano]

Are you running out of memory perhaps? Could you provide cat /proc/meminfo ?

[MaoPann]

MemTotal: 8124336 kB MemFree: 5632648 kB MemAvailable: 6103652 kB Buffers: 58432 kB Cached: 608900 kB SwapCached: 0 kB Active: 1505616 kB Inactive: 479092 kB Active(anon): 1318760 kB Inactive(anon): 6852 kB Active(file): 186856 kB Inactive(file): 472240 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 969960 kB SwapFree: 969960 kB Dirty: 28216 kB Writeback: 0 kB AnonPages: 1317392 kB Mapped: 250300 kB Shmem: 8240 kB KReclaimable: 66204 kB Slab: 147760 kB SReclaimable: 66204 kB SUnreclaim: 81556 kB KernelStack: 13924 kB PageTables: 40656 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 5032128 kB Committed_AS: 5451100 kB VmallocTotal: 34359738367 kB VmallocUsed: 29776 kB VmallocChunk: 0 kB Percpu: 44032 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB FileHugePages: 0 kB FilePmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 167744 kB DirectMap2M: 5074944 kB DirectMap1G: 5242880 kB

[cardigliano]

Since your system does not have plenty of physical memory, please try reducing the buffer size using this configuration:

--interface=ens33 --dump-directory=/storage/n2disk/pcap --timeline-dir=/storage/n2disk/timeline --disk-limit=512 # --max-file-len=500 --buffer-len=2000 --max-file-duration=60 --index --snaplen=1536 # --writer-cpu-affinity=0 --reader-cpu-affinity=1 --compressor-cpu-affinity=2,3 --index-on-compressor-threads # -u=ntopng --zmq=tcp://127.0.0.1:5556 --zmq-probe-mode --zmq-export-flows

[MaoPann]

Ihave changed the configuration,but I was unable to receive flows data from n2disk and reported the following error.

[cardigliano]

Please note the --disk-limit unit it MBytes, please use something multiple of the file size, like --disk-limit=10000 (or a % of the disk space with --disk-limit=50%)

[MaoPann]

Hi , Now I can normally use n2disk + ntopng , 1、it seems that I can't pcap files in advance, and I can't configure external traffic recording providers。

My profile is as follows： n2disk.conf

--interface=ens160u1 --dump-directory=/var/lib/ntopng/n2disk/pcap --timeline-dir=/var/lib/ntopng/n2disk/timeline --disk-limit=95% # --max-file-len=256 --buffer-len=256 --max-file-duration=60 --index --snaplen=1536

-b=512

# --writer-cpu-affinity=0 --reader-cpu-affinity=1 --compressor-cpu-affinity=2,3 --index-on-compressor-threads # -u=ntopng --zmq=tcp://127.0.0.1:1234 --zmq-probe-mode --zmq-export-flows

ntopng.conf

-i=tcp://*:1234c -F=nindex -G=/var/run/ntopng.pid

[cardigliano]

What do you mean with "I can't pcap files in advance"? What is the issue you are experiencing about "I can't configure external traffic recording providers" exactly?

[MaoPann]

Full description: I want to use n2disk+ntopng to monitor network traffic. I need to extract the pcap file of historical traffic. I used the configuration method linked below.

1、I can't extract the traffic pcap file 2、I can't find this option

[cardigliano]

Are you running n2disk as a service? Please check https://www.ntop.org/guides/n2disk/how_to_start.html#configuration-file-format In short:

• Create /etc/n2disk/n2disk-ens160u1.conf with the configuration
• Run the service: systemctl enable n2disk@ens160u1 systemctl start n2disk@ens160u1

[MaoPann]

I did, but I couldn't extract pcap

[MaoPann]

[cardigliano]

@MaoPann are you still experiencing this issue? Any chance I can somehow connect to this box to review the whole configuration?

[MaoPann]

@cardigliano yes, I hope to use the following methods to continuously store 10G-40Gbps traffic, use ntopng for monitoring, and extract pcapfile. If I monitor a span port, please give me a correct configuration file case, so that I can change it according to my actual needs, because my own test has not been successful. ![Uploading image.png…]()

[cardigliano]

This is a sample configuration for capturing 10-20Gbps on a 6-cores CPU:

--interface=zc:eno1 --dump-directory=/storage/n2disk/pcap --timeline-dir=/storage/n2disk/timeline --disk-limit=80% --max-file-len=1024 --buffer-len=8192 --index -S=0 -c=1 -w=2 -z=3,4,5 -Z

[MaoPann]

@cardigliano This is just the configuration of n2disk. I need to use it with ntopng. How should I configure it? In addition to storing traffic, we also need to monitor it and extract pcap files stored in n2disk in ntopng.In addition, n2disk requires multiple RSS queues

[cardigliano]

I assumed you already knew how to configure an external provider in ntopng (see https://www.ntop.org/guides/ntopng/using_with_other_tools/n2disk.html#external-traffic-recording-providers) Otherwise as I said I can connect to this box and review the whole configuration, if possible.

[MaoPann]

Yes, I know how to configure external traffic recording providers, but this means that I need to use ntopng - i zc: eno1, so I will use ZC command twice in ntopng and n2disk.

[cardigliano]

@MaoPann there are two options:

1. if the traffic rate (pps) is not that high, you should use a standard/kernel driver (no zc:) both in n2disk and ntopng
2. otherwise, you should use zc in n2disk, and forward flow metadata to ntopng as explained at https://www.ntop.org/guides/ntopng/using_with_other_tools/n2disk.html#zc-fpga-support