vel21ripn / nDPI

Open Source Deep Packet Inspection Software Toolkit
http://www.ntop.org
GNU Lesser General Public License v3.0
119 stars 59 forks source link

Compile error "stdbool.h no such file or directory" #134

Closed ewildgoose closed 1 year ago

ewildgoose commented 2 years ago

I've just updated to git revision 1367ad9, ie branch flow-4 latest. I corrected the kernel compile with the pull request I sent separately, but this leaves a compile error as follows:

# make MODULES_DIR=/lib/modules/5.17.4 KERNEL_DIR=/usr/src/linux
make -C ipt
make[1]: Entering directory '/var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/ipt'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src'
grep: /usr/src/linux/source/include/linux/skbuff.h: No such file or directory
$xt_ndpi-y is [main.o ndpi_strcol.o ndpi_proc_parsers.o ndpi_proc_generic.o ndpi_proc_info.o  ndpi_proc_flow.o ndpi_proc_hostdef.o ndpi_proc_ipdef.o ../libre/regexp.o ../../src/lib/third_party/src/ndpi_md5.o ../../src/lib/third_party/src/ndpi_sha1.o ../../src/lib/third_party/src/ahocorasick.o ../../src/lib/third_party/src/libcache.o ../../src/lib/third_party/src/ndpi_patricia.o ../../src/lib/third_party/src/gcrypt_light.o ../../src/lib/third_party/src/btlib.o ../../src/lib/ndpi_main.o ../../src/lib/ndpi_utils.o ../../src/lib/ndpi_serializer.o ../../src/lib/ndpi_geoip.o ../../src/lib/protocols/*.o] -I/../../src/include -I/../../src/lib -I/../libre -I/../../src/lib/third_party/include  -DHAVE_CONFIG_H -DNDPI_LIB_COMPILATION -DOPENDPI_NETFILTER_MODULE -DNDPI_DETECTION_SUPPORT_IPV6 -g -Wshadow-local
\
make  -C /usr/src/linux M=$PWD modules -w;
make[2]: Entering directory '/usr/src/linux-tpb'
grep: /usr/src/linux/source/include/linux/skbuff.h: No such file or directory
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/main.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/ndpi_strcol.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/ndpi_proc_parsers.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/ndpi_proc_generic.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/ndpi_proc_info.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/ndpi_proc_flow.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/ndpi_proc_hostdef.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/ndpi_proc_ipdef.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/ndpi_md5.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/ndpi_sha1.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/ahocorasick.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/libcache.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/ndpi_patricia.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/gcrypt_light.o
In file included from /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/gcrypt_light.c:57:
/var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/gcrypt/aes.c: In function 'aes_gen_tables':
/var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/gcrypt/aes.c:143:1: warning: the frame size of 2056 bytes is larger than 2048 bytes [-Wframe-larger-than=]
  143 | }
      | ^
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/third_party/src/btlib.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/ndpi_main.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/ndpi_utils.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/ndpi_serializer.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/afp.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/aimini.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/ajp.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/amazon_video.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/among_us.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/amqp.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/apple_push.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/applejuice.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/armagetron.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/avast_securedns.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/ayiya.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/bgp.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/bittorrent.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/bjnp.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/capwap.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/cassandra.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/checkmk.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/ciscovpn.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/citrix.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/coap.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/collectd.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/corba.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/cpha.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/crossfire.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/csgo.o
  CC [M]  /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/dcerpc.o
/var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/dcerpc.c:29:10: fatal error: stdbool.h: No such file or directory
   29 | #include <stdbool.h>
      |          ^~~~~~~~~~~
compilation terminated.
make[3]: *** [scripts/Makefile.build:288: /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src/../../src/lib/protocols/dcerpc.o] Error 1
make[2]: *** [Makefile:1831: /var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src] Error 2
make[2]: Leaving directory '/usr/src/linux-tpb'
make[1]: *** [Makefile:43: modules] Error 2
make[1]: Leaving directory '/var/tmp/portage/net-firewall/nDPI-netfilter-20220419/work/nDPI-netfilter-20220419/ndpi-netfilter/src'
make: *** [Makefile:3: all] Error 2

However, I seem to have this file on my system:

# find /usr/ -name stdbool.h
/usr/include/unistring/stdbool.h
/usr/include/textstyle/stdbool.h
/usr/include/x86_64-gentoo-linux-musl/textstyle/stdbool.h
/usr/include/stdbool.h
/usr/lib/gcc/x86_64-gentoo-linux-musl/10.3.0/include/stdbool.h
/usr/lib/gcc/x86_64-gentoo-linux-musl/10.3.0/include/g++-v10/tr1/stdbool.h
/usr/lib/gcc/x86_64-gentoo-linux-musl/11.2.0/include/stdbool.h
/usr/lib/gcc/x86_64-gentoo-linux-musl/11.2.0/include/g++-v11/tr1/stdbool.h

Help! I can't figure out what I've busted here?

For background, this is built via gentoo and it's build system. I think I have all needed dependencies, but I'm not really a C guy, and unsure why my includes are busted at this point? How might I go about understanding and debugging this?? (Edited to show the whole compile when run manually from the command line)

Thanks

ewildgoose commented 2 years ago

Thoughts: Seems as though the src/lib/Makefile.am isn't including /usr/include in it's -I stanza

I think I'm a bit confused on what SHOULD be happening here? Where is this include on other systems, surely it can't only be me noticing this?

vel21ripn commented 2 years ago

For kernel modules uses option "-nostdinc". Look for a hint in the nDPI/src/protocols/.dcerpc.o.cmd file.

I'm still fixing the errors made when adding new functionality. I will apply the fix for 5.17 a little later.

ewildgoose commented 2 years ago

Thanks for the hint. I have had a quite look at this, but I think the hint isn't enough for me to figure out what I did wrong?

I don't have the dir you mention. Finding the protocols dir I don't have very many .o.cmd files and I don't have .dcerpc.o.cmd file?

I did a grep through the code for -nostdinc and I don't find any clues. Running "make blah -nostdinc" seems to lead to it not compiling anything at all...

Bleaugh. I'm a bit confused. Any chance of a further leg up please?

vel21ripn commented 2 years ago

Find any .o.cmd file in the nDPI/src/protocols/ directory. In it you will find all the options with which the compiler is launched, including all the directories in which the compiler will look for header files are listed there. Are there any of these directories that contain the stdbool.h file?

There is a suspicion that you have some temporary obsolete file. Doing "make distclean" on the nDPI directory. And then run the command:

find  -type f \( -name '*.o' -o -name '*.ko' -o -name '*.o.cmd' -o -name '*.ko.cmd' \
        -o -name '*.mod' -o -name '*.mod.c' -o -name '*.mod.o' \
        -o -name .Module.symvers.cmd -o -name .modules.order.cmd \
        -o -name Module.symvers -o -name modules.order \
        -o -name '*.mod.cmd' -o -name .cache.mk \) -delete -print
ewildgoose commented 2 years ago

Can you evaluate the following patch please. I concede I'm not sure what I'm doing, so...

Reading docs it seems as though for using bool type in kernel space we need "#include <linux/types.h>"? I'm not clear why everything is compiling for you, but at least on my musl system, I'm having trouble getting stdbool.h being in the search path

This patch works for me on an amd64 musl system. Could someone confirm it does no harm on other systems? (I'll shortly test on i386 and arm 32bit)

Thanks

--- a/src/lib/protocols/dcerpc.c        2022-05-31 18:47:46.908279592 +0000
+++ b/src/lib/protocols/dcerpc.c        2022-05-31 18:50:26.575524062 +0000
@@ -26,7 +26,8 @@
 #define NDPI_CURRENT_PROTO NDPI_PROTOCOL_RPC

 #include "ndpi_api.h"
-#include <stdbool.h>
+/* Define bool for kernel compile */
+#include <linux/types.h>

 static void ndpi_int_dcerpc_add_connection(struct ndpi_detection_module_struct
                                             *ndpi_struct, struct ndpi_flow_struct *flow)
vel21ripn commented 2 years ago

Thanks! Fixed in 9f9138c5aa0eda811822a756535aab9b566e1c44