natevents sending flows only if "conntrack -E" is running

[JoyHusky]

ipt_NETFLOW не шлёт flow-ы на указаный в destination IP, до тех пор, пока не запущу conntrack -E. В счётчике "Total x pkts" соответственно тоже по нулям, пока не запущу вышеуказанную утилиту.

uname -a Linux nat1.pvl 6.0.10-arch2-1 #1 SMP PREEMPT_DYNAMIC Sat, 26 Nov 2022 16:51:18 +0000 x86_64 GNU/Linux

cat /proc/net/stat/ipt_netflow ipt_NETFLOW 2.6-12-g40fefb2, srcversion D591EC4E9D437B9AA55C498; llist nel

cat /etc/modprobe.d/ipt_NETFLOW.conf options ipt_NETFLOW hashsize=160000 sndbuf=1000000 destination=46.36.x.x:9999 protocol=5 natevents=1

установка:

$ ./configure --enable-natevents
Module version: 2.6-12-g40fefb2
Kernel version: 6.0.10-arch2-1 (uname)
Kernel sources: /lib/modules/6.0.10-arch2-1/build (found)
Checking for presence of include/linux/netfilter.h... Yes
netfilter.h uses CONFIG_NF_NAT_NEEDED... No
Checking for presence of include/linux/llist.h... Yes
Checking for presence of include/linux/grsecurity.h... No
Iptables binary version: 1.8.8 (legacy) (detected from /usr/bin/iptables)
pkg-config for version 1.8.8 (legacy) exists: No (reported: 1.8.8)
Check for working gcc: Yes (gcc)
Checking for presence of xtables.h... Yes
Searching for iptables-1.8.8 (legacy) sources..
! Can not find iptables source directory, you may try setting it with --ipt-src=
! This is not fatal error, yet. Will be just using default include dir.
Iptables include flags: none (default)
Iptables module path: /usr/lib/xtables (from libxtables.so, from binary)
Searching for net-snmp-config... Yes /usr/bin/net-snmp-config
Searching for net-snmp agent... Yes.
Checking for DKMS... Yes.
Creating Makefile.. done.

  If you need some options enabled run ./configure --help
  Now run: make all install

$ make all
./gen_compat_def > compat_def.h-
Test function xt_family linux/netfilter_ipv4/ip_tables.h  declared
Test struct timeval linux/ktime.h  undeclared
egrep: warning: egrep is obsolescent; using grep -E
Test struct proc_ops linux/proc_fs.h  declared
Test function synchronize_sched linux/rcupdate.h  undeclared
egrep: warning: egrep is obsolescent; using grep -E
Test function nf_bridge_info_get linux/netfilter_bridge.h  declared
Test struct vlan_dev_priv linux/if_vlan.h  declared
Test function put_unaligned_be24 asm/unaligned.h  declared
Test function totalram_pages linux/mm.h  declared
Test symbol totalram_pages linux/mm.h  declared
Test member nf_ct_event_notifier.ct_event net/netfilter/nf_conntrack_ecache.h  declared
mv compat_def.h- compat_def.h
Compiling 2.6-12-g40fefb2 for kernel 6.0.10-arch2-1
make -C /lib/modules/6.0.10-arch2-1/build M=/home/joy/build/ipt-netflow modules
make[1]: вход в каталог «/usr/lib/modules/6.0.10-arch2-1/build»
  CC [M]  /home/joy/build/ipt-netflow/ipt_NETFLOW.o
  MODPOST /home/joy/build/ipt-netflow/Module.symvers
  CC [M]  /home/joy/build/ipt-netflow/ipt_NETFLOW.mod.o
  LD [M]  /home/joy/build/ipt-netflow/ipt_NETFLOW.ko
  BTF [M] /home/joy/build/ipt-netflow/ipt_NETFLOW.ko
make[1]: выход из каталога «/usr/lib/modules/6.0.10-arch2-1/build»
gcc  -O2 -Wall -Wunused -DXTABLES   -fPIC -o libipt_NETFLOW_sh.o -c libipt_NETFLOW.c
gcc -shared -o libipt_NETFLOW.so libipt_NETFLOW_sh.o
gcc  -O2 -Wall -Wunused -DXTABLES   -fPIC -o libip6t_NETFLOW_sh.o -c libipt_NETFLOW.c
gcc -shared -o libip6t_NETFLOW.so libip6t_NETFLOW_sh.o
gcc -fPIC -shared -o snmp_NETFLOW.so snmp_NETFLOW.c -lnetsnmp
rm libipt_NETFLOW_sh.o libip6t_NETFLOW_sh.o

$ sudo make install
 *
make -C /lib/modules/6.0.10-arch2-1/build M=/home/joy/build/ipt-netflow modules_install INSTALL_MOD_PATH=
make[1]: вход в каталог «/usr/lib/modules/6.0.10-arch2-1/build»
  INSTALL /lib/modules/6.0.10-arch2-1/extra/ipt_NETFLOW.ko
  SIGN    /lib/modules/6.0.10-arch2-1/extra/ipt_NETFLOW.ko
At main.c:167:
- SSL error:FFFFFFFF80000002:system library::No such file or directory: crypto/bio/bss_file.c:67
- SSL error:10000080:BIO routines::no such file: crypto/bio/bss_file.c:75
sign-file: ./certs/signing_key.pem
  ZSTD    /lib/modules/6.0.10-arch2-1/extra/ipt_NETFLOW.ko.zst
  DEPMOD  /lib/modules/6.0.10-arch2-1
make[1]: выход из каталога «/usr/lib/modules/6.0.10-arch2-1/build»
/sbin/depmod -a
 *
install -D libipt_NETFLOW.so /usr/lib/xtables/libipt_NETFLOW.so
install -D libip6t_NETFLOW.so /usr/lib/xtables/libip6t_NETFLOW.so
 *
Installing into DKMS...
! Installing 2.6-12-g40fefb2 into DKMS...
Creating symlink /var/lib/dkms/ipt-netflow/2.6-12-g40fefb2/source -> /usr/src/ipt-netflow-2.6-12-g40fefb2
 *
install -D IPT-NETFLOW-MIB.my /usr/share/snmp/mibs/IPT-NETFLOW-MIB.my
install -D snmp_NETFLOW.so /usr/lib/snmp/dlmod/snmp_NETFLOW.so
egrep: warning: egrep is obsolescent; using grep -E
 *  (snmpd needs restart for changes to take effect.)

[JoyHusky]

Этот же сервер, но ядро по-старее, всё работает:

a@a# uname -a Linux pavl-nat-1 5.10.8-arch1-1 #1 SMP PREEMPT Sun, 17 Jan 2021 22:07:13 +0000 x86_64 GNU/Linux

a@a# head -n7 /proc/net/stat/ipt_netflow ipt_NETFLOW 2.6-12-g40fefb2, srcversion D591EC4E9D437B9AA55C498; aggr llist mac nel Protocol version 5 (netflow) Timeouts: active 1800s, inactive 15s. Maxflows 2000000 Natevents enabled, count start 296, stop 281. Flows: active 0 (peak 0 reached 0d0h4m ago), mem 1250K, worker delay 30/300 [1..30] (20 ms, 0 us, 0:0 0 [cpu12]). Hash: size 160000 (mem 1250K), metric 1.00 [1.00, 1.00, 1.00]. InHash: 0 pkt, 0 K, InPDU 0, 0. Rate: 0 bits/sec, 0 packets/sec; Avg 1 min: 0 bps, 0 pps; 5 min: 0 bps, 0 pps

[1223421]

natevents в этом модуле не работают на ядрах >=6, подтверждаю, пытался найти проблему, но не осилил... Единственное что удалось понять, то что это условие с NULL не срабатывает, на 5 ядре - срабатывает:

static int set_notifier_cb(NET_STRUCT) 
 {
         struct nf_ct_event_notifier *notifier;

         notifier = rcu_dereference(nf_conntrack_event_cb);
         if (notifier == NULL) {

[vvfedorenko]

в ядро 5.19 добавили новое значение для net.netfilter.nf_conntrack_events=2 и сделали его значением по умолчанию. оно означает, что пока нет netlink соединений, слушающих nat events - не создавать собственно nat events. Поэтому со свежими ядрами при установках по умолчанию модуль не работает. sysctl net.netfilter.nf_conntrack_events=1 решает проблему

[vvfedorenko]

@aabc mark it resolved

[JoyHusky]

Resolved by set sysctl net.netfilter.nf_conntrack_events=1

thx vvfedorenko