aabc
commented
2 years ago It should show error message from compiler that it cannot interpret.
Closed xtaran closed 2 years ago
aabc
commented
2 years ago It should show error message from compiler that it cannot interpret.
xtaran
commented
2 years ago TL;DR: I think this is actually a regression in the Linux "Kbuild" part of kernel 5.13, namely in commit torvalds/linux@b0030af53a74a08c77ea11d3888da21542af2d0e. I have currently no idea if this is fixable in ipt_NETFLOW.
How I came to that conclusion (long reasoning with all my steps written down as I made them):
It should show error message from compiler that it cannot interpret.
Ok, so I tried it without the Debian package around it and without compiling for a non-running (but installed) kernel:
I took a Raspberry Pi 4, downloaded https://raspi.debian.net/verified/20210718_raspi_4_bullseye.img.xz (via https://raspi.debian.net/tested-images/), applied all updates, added the Debian Experimental repo by adding this line to /etc/apt/sources.list and ran apt update again:
deb http://deb.debian.org/debian experimental main contrib non-freeThen I installed these packages:
linux-image-5.13.0-trunk-arm64 and linux-headers-5.13.0-trunk-arm64 (only those two are from the Debian Experimental APT repo)build-essential (for make, gcc, etc.)libxtables-dev (not iptables-dev as advised by the ./configure output — that one no more exists in Debian Unstable since December 2019 and was last part of the Debian 10 "Buster" release from 2 years ago—only as transitional package. And it was a transitional package since Debian 9 "Stretch", released 4 years ago.)dkmspkg-config(of course including all the needed dependencies)
I then downloaded https://github.com/aabc/ipt-netflow/archive/refs/tags/v2.6.tar.gz, unpacked it into the directory ipt-netflow-2.6, changed to that directory and ran ./configure and make. But the result is the same:
~/ipt-netflow-2.6$ ./configure && make
Module version: 2.6
Kernel version: 5.13.9 (proc)
Kernel sources: /lib/modules/5.13.0-trunk-arm64/build (found)
Checking for presence of include/linux/netfilter.h... Yes
netfilter.h uses CONFIG_NF_NAT_NEEDED... No
Checking for presence of include/linux/llist.h... Yes
Checking for presence of include/linux/grsecurity.h... No
Iptables binary version: no iptables binary found
Xtables version: 1.8.7 (detected from /usr/bin/pkg-config)
Check for working gcc: Yes (gcc)
Checking for presence of xtables.h... Yes
Iptables include flags: (pkg-config)
Iptables module path: /usr/lib/aarch64-linux-gnu/xtables (pkg-config)
Searching for net-snmp-config... No.
Searching for net-snmp agent... No.
Assuming you don't want net-snmp agent support.
Otherwise do: apt-get install snmpd libsnmp-dev
Checking for DKMS... Yes.
Creating Makefile.. done.
If you need some options enabled run ./configure --help
Now run: make all install
./gen_compat_def > compat_def.h
Test symbol xt_family linux/netfilter_ipv4/ip_tables.h undeclared
Error: unexpected error from compiler
make -s -B -C /lib/modules/5.13.0-trunk-arm64/build M=/home/abe/ipt-netflow-2.6/cc-test-build modules
sh: 0: cannot open /usr/src/linux-headers-5.13.0-trunk-common/scripts/modules-check.sh: No such file
make[2]: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:1796: modules_check] Error 2
make[1]: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:232: __sub-make] Error 2
make: *** [Makefile:30: compat_def.h] Error 3(The last four or five lines looked not relevant to as I considered them to be follow-up errors, but as I noticed later, they seem at least related.)
So I ran just ./gen_compat_def:
~/ipt-netflow-2.6$ ./gen_compat_def
// Autogenerated for /lib/modules/5.13.0-trunk-arm64/build
Test symbol xt_family linux/netfilter_ipv4/ip_tables.h undeclared
#undef HAVE_XT_FAMILY
// xt_family is undeclared in <linux/netfilter_ipv4/ip_tables.h>. Compile:
// #include <linux/module.h>
// #include <linux/netfilter_ipv4/ip_tables.h>
// MODULE_LICENSE("GPL");
// void *test = xt_family;
// Output:
// make -s -B -C /lib/modules/5.13.0-trunk-arm64/build M=/home/abe/ipt-netflow-2.6/cc-test-build modules
// sh: 0: cannot open /usr/src/linux-headers-5.13.0-trunk-common/scripts/modules-check.sh: No such file
// make[1]: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:1796: modules_check] Error 2
// make: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:232: __sub-make] Error 2
Error: unexpected error from compiler
make -s -B -C /lib/modules/5.13.0-trunk-arm64/build M=/home/abe/ipt-netflow-2.6/cc-test-build modules
sh: 0: cannot open /usr/src/linux-headers-5.13.0-trunk-common/scripts/modules-check.sh: No such file
make[1]: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:1796: modules_check] Error 2
make: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:232: __sub-make] Error 2I hope that this is more helpful.
I also cross-checked: I don't have a file called modules-check.sh on any of my systems where ipt_NETFLOW has successfully built. With kernel 5.10 this looks very different:
[…]-netflow/ipt-netflow → ./gen_compat_def
// Autogenerated for /lib/modules/5.10.0-8-amd64/build
Test function xt_family linux/netfilter_ipv4/ip_tables.h declared
// xt_family is declared in <linux/netfilter_ipv4/ip_tables.h>
#define HAVE_XT_FAMILY
Test struct timeval linux/ktime.h undeclared
#undef HAVE_TIMEVAL
[…](Succeeds in the end.)
So I diffed /usr/src/linux-headers-5.10.0-8-common/include/linux/netfilter_ipv4/ip_tables.h and /usr/src/linux-headers-5.13.0-trunk-common/include/linux/netfilter_ipv4/ip_tables.h and they look very differently — as if they never were the same file.
xt_family is though declared in both kernel versions in the file …/include/linux/netfilter/x_tables.h:
→ dgrep xt_family linux-headers-5.10.0-8-common linux-headers-5.13.0-trunk-common
/usr/src/linux-headers-5.10.0-8-common/include/linux/netfilter/x_tables.h:static inline u_int8_t xt_family(const struct xt_action_param *par)
/usr/src/linux-headers-5.13.0-trunk-common/include/linux/netfilter/x_tables.h:static inline u_int8_t xt_family(const struct xt_action_param *par)Interestingly, <linux/netfilter/x_tables.h> is only directly #included in kernel 5.13, but not in kernel 5.10, despite the compilation result looks the opposite way.
Next step deeper into that rabbit hole: Compile that test.c manually, without the -s for silent:
→ cat test.c
#include <linux/module.h>
#include <linux/netfilter_ipv4/ip_tables.h>
MODULE_LICENSE("GPL");
void *test = xt_family;
→ cat Makefile
obj-m = test.o
→ make -B -C /lib/modules/5.13.0-trunk-amd64/build M=`pwd` modules
make: Entering directory '/usr/src/linux-headers-5.13.0-trunk-amd64'
CC [M] /home/abe/ipt-netflow/tmp/test.o
sh: 0: cannot open /usr/src/linux-headers-5.13.0-trunk-common/scripts/modules-check.sh: No such file
make[1]: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:1796: modules_check] Error 2
make: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:232: __sub-make] Error 2
make: Leaving directory '/usr/src/linux-headers-5.13.0-trunk-amd64'
-rw-r--r-- 1 abe abe 15 Aug 18 00:34 Makefile
-rw-r--r-- 1 abe abe 4 Mar 20 2018 foo
-rw-r--r-- 1 abe abe 34 Aug 18 00:38 modules.order
-rw-r--r-- 1 abe abe 117 Aug 18 00:31 test.c
-rw-r--r-- 1 abe abe 34 Aug 18 00:38 test.mod
-rw-r--r-- 1 abe abe 280704 Aug 18 00:38 test.o
So it actually compiled test.o successfully. Accordingly I started diffing /usr/src/linux-headers-5.*-common/Makefile as this seems what calls this non-existent file. And the diff was rather huge, too. And the part where the non-existent file gets called got moved around.
Next step was to temporarily move /usr/src/linux-headers-5.13.0-trunk-common/Makefile away and replace it with a symlink to /usr/src/linux-headers-5.10.0-8-common/Makefile. Yeah, I didn't expect it to work, but it now came much farther:
# cat /var/lib/dkms/ipt-netflow/2.6/build/make.log
DKMS make.log for ipt-netflow-2.6 for kernel 5.13.0-trunk-amd64 (x86_64)
Wed Aug 18 00:58:08 CEST 2021
./gen_compat_def > compat_def.h
Test symbol xt_family linux/netfilter_ipv4/ip_tables.h declared
Test struct timeval linux/ktime.h undeclared
Test struct proc_ops linux/proc_fs.h declared
Test symbol synchronize_sched linux/rcupdate.h undeclared
Test symbol nf_bridge_info_get linux/netfilter_bridge.h declared
Test struct vlan_dev_priv linux/if_vlan.h declared
Compiling 2.6 for kernel 5.13.9
make -C /lib/modules/5.13.0-trunk-amd64/build M=/var/lib/dkms/ipt-netflow/2.6/build modules
make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
make[1]: Entering directory '/usr/src/linux-headers-5.13.0-trunk-amd64'
arch/x86/Makefile:148: CONFIG_X86_X32 enabled but no binutils support
CC [M] /var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:96:4: warning: #warning "Requested physdev is not compiled." [-Wcpp]
96 | # warning "Requested physdev is not compiled."
| ^~~~~~~
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c: In function ‘nf_seq_show’:
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:762:39: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]
762 | seq_printf(seq, " Flows selected %lu, discarded %lu.",
| ~~^
| |
| long unsigned int
| %llu
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:762:54: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 4 has type ‘s64’ {aka ‘long long int’} [-Wformat=]
762 | seq_printf(seq, " Flows selected %lu, discarded %lu.",
| ~~^
| |
| long unsigned int
| %llu
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:766:39: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]
766 | seq_printf(seq, " Flows selected %lu.", atomic64_read(&flows_selected));
| ~~^
| |
| long unsigned int
| %llu
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c: In function ‘get_template’:
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:3427:17: warning: taking address of packed member of ‘struct data_template’ may result in an unaligned pointer value [-Waddress-of-packed-member]
3427 | hlist_add_head(&tpl->hlist, &templates_hash[hash]);
| ^~~~~~~~~~~
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c: In function ‘pdu_add_template’:
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:3498:11: warning: taking address of packed member of ‘struct data_template’ may result in an unaligned pointer value [-Waddress-of-packed-member]
3498 | fields = tpl->fields;
| ^~~
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c: In function ‘export_stat_st_ts’:
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:3957:62: warning: ‘tpl’ may be used uninitialized in this function [-Wmaybe-uninitialized]
3957 | pdu_flowset->length = htons(ntohs(pdu_flowset->length) + tpl->rec_size);
| ^~
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c: In function ‘netflow_export_flow_tpl’:
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:3860:21: warning: ‘tpl’ may be used uninitialized in this function [-Wmaybe-uninitialized]
3860 | ptr += tpl->fields[i++];
| ~~~~~~~~~~~^~~~~
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c: In function ‘netflow_scan_and_export’:
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:4235:62: warning: ‘tpl’ may be used uninitialized in this function [-Wmaybe-uninitialized]
4235 | pdu_flowset->length = htons(ntohs(pdu_flowset->length) + tpl->rec_size);
| ^~
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.c:4179:24: note: ‘tpl’ was declared here
4179 | struct data_template *tpl;
| ^~~
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: get_template()+0x3f5: indirect jump found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: usock_connect()+0x6e: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: usock_connect()+0xf1: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: usock_connect()+0x1ba: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: usock_connect()+0x252: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: alloc_record_tpl()+0x29e: indirect jump found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: alloc_record_tpl()+0x2a8: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: alloc_record_tpl()+0x2d0: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: alloc_record_tpl()+0x3c1: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: alloc_record_tpl()+0x3e0: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: export_stat_st_ts()+0x84: indirect jump found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: export_stat_st_ts()+0x15d: indirect jump found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: export_stat_st_ts()+0x175: indirect jump found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_conntrack_event()+0x4f: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_export_flow_tpl()+0x82: indirect jump found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0x28d: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0x59e: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0x68c: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0x6be: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0x742: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0xc06: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0xc27: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: netflow_scan_and_export()+0xc84: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: promisc_rcv()+0x3fc: indirect call found in RETPOLINE build
/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.o: warning: objtool: promisc_rcv()+0x764: indirect call found in RETPOLINE build
MODPOST /var/lib/dkms/ipt-netflow/2.6/build/Module.symvers
ERROR: modpost: "mcount" [/var/lib/dkms/ipt-netflow/2.6/build/ipt_NETFLOW.ko] undefined!
make[3]: *** [/usr/src/linux-headers-5.13.0-trunk-common/scripts/Makefile.modpost:150: /var/lib/dkms/ipt-netflow/2.6/build/Module.symvers] Error 1
make[3]: *** Deleting file '/var/lib/dkms/ipt-netflow/2.6/build/Module.symvers'
make[2]: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:1749: modules] Error 2
make[1]: *** [/usr/src/linux-headers-5.13.0-trunk-common/Makefile:185: __sub-make] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.13.0-trunk-amd64'
make: *** [Makefile:27: ipt_NETFLOW.ko] Error 2So for me this now looks as if there is a regression in the top-level Makefile of the Linux kernel 5.13 (again given that it was said to work with kernel 5.12). Looks as if torvalds/linux@b0030af53a74a08c77ea11d3888da21542af2d0e could be the culprit.
aabc
commented
2 years ago Why you tested on arm and not on x86_64? And why v2.6 and not master? I test on gcc 10 (no ton Debian though) and it builds OK for Linux v5.13 (both master and v2.6).
xtaran
commented
2 years ago Why you tested on arm
Because it's the easiest way for me to toy around with different kernels. It's easier and faster to bootstrap than a VM for me: just a dd onto an SD card). And it shouldn't make any difference. If it does nevertheless, it's a bug.
and not on x86_64?
I did that as well. Just not with a running kernel 5.13 as I'm not keen on rebooting my workstation that often.
And why v2.6 and not master?
Because I'm packaging 2.6 for Debian and not master. And because a released versions is a declaration of stability while building from a git branch does not need to be working depending on the way of development.
I test on gcc 10 (not on Debian though) and it builds OK for Linux v5.13 (both master and v2.6).
That it wasn't on Debian was the difference. It turned out to indeed be a bug in Debian's most recent kernel package. 2.6 compiles fine now with DKMS on Debian again. Sorry for the noise. Hence closing this issue herewith.
aabc
commented
2 years ago Thanks for update!
I just tried to build ipt_NETFLOW with DKMS against kernel 5.13.9 from Debian Experimental (package version 5.13.9-1~exp2, likely will soon show up in Debian Unstable, too) and it failed as follows:
Compiler was GCC 10.2.1 from Debian Unstable (currently still the same version as in Debian 11 — which was just released yesterday):
Note: Since this is the first kernel uploaded to Debian newer than kernel 5.10.x as shipped with Debian 11, I'm not 100% sure if this is a just not yet reported compatibility issue in ipt_NETFLOW with kernel 5.13 in general or a regression in Debian's kernel packages after quite a while (half a year) of no invasive changes at all. At least I've found no hints about a potential iptables removal or so in the changelogs of Debian or the kernel 5.13. (ipt_NETFLOW 2.6 was reported to successfully compile against kernel 5.12, so I didn't bother to look up earlier kernel changes.)