build issues on ubuntu 21.10

[mathiasxx]

Probably user error, but figure I'll report this just in case. My system

• Ubuntu 21.10
• Linux ubunuc 5.13.0-27-generic #29-Ubuntu SMP Wed Jan 12 17:36:47 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
• gcc (Ubuntu 11.2.0-7ubuntu2) 11.2.0
• GNU ld (GNU Binutils for Ubuntu) 2.37
• GNU Make 4.3

After doing the follow dependency related installs:

• apt-get install libnl-3-dev
• apt-get install libnl-genl-3-dev
• apt-get install libreadline-dev
• apt-get install libpcap-dev
• apt-get install binutils-dev

autogen and configure run fine, but make fails with:

make  all-recursive
make[1]: Entering directory '/home/mathias/src/dropwatch'
Making all in src
make[2]: Entering directory '/home/mathias/src/dropwatch/src'
gcc -g -Wall -Werror -I/usr/include/libnl3 -I/usr/include/libnl3 -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -g -O2 -lnl-3 -lnl-genl-3 -lnl-3 -lreadline -lpcap -lbfd  -o dropwatch main.o lookup.o lookup_kas.o lookup_bfd.o  
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `handle_dm_config_new_msg':
/home/mathias/src/dropwatch/src/main.c:561: undefined reference to `genlmsg_parse'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:570: undefined reference to `nla_get_u8'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:582: undefined reference to `nla_get_u32'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:585: undefined reference to `nla_get_u32'
/usr/bin/ld: main.o: in function `setup_netlink_socket':
/home/mathias/src/dropwatch/src/main.c:170: undefined reference to `nl_socket_alloc'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:172: undefined reference to `genl_connect'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:174: undefined reference to `genl_ctrl_resolve'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:183: undefined reference to `nl_close'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:184: undefined reference to `nl_socket_free'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:186: undefined reference to `nl_socket_alloc'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:187: undefined reference to `nl_join_groups'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:189: undefined reference to `nl_connect'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:194: undefined reference to `nl_close'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:195: undefined reference to `nl_socket_free'
/usr/bin/ld: main.o: in function `alloc_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:210: undefined reference to `nlmsg_alloc'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:211: undefined reference to `genlmsg_put'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `recv_netlink_message':
/home/mathias/src/dropwatch/src/main.c:280: undefined reference to `nl_recv'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:325: undefined reference to `nlmsg_data'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `recv_netlink_message':
/home/mathias/src/dropwatch/src/main.c:307: undefined reference to `nlmsg_data'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `process_rx_message':
/home/mathias/src/dropwatch/src/main.c:354: undefined reference to `nlmsg_data'
/usr/bin/ld: main.o: in function `print_nested_hw_entry':
/home/mathias/src/dropwatch/src/main.c:366: undefined reference to `nla_parse_nested'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:374: undefined reference to `nla_get_string'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:374: undefined reference to `nla_get_u32'
/usr/bin/ld: main.o: in function `print_nested_hw_entries':
/home/mathias/src/dropwatch/src/main.c:384: undefined reference to `nla_data'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:384: undefined reference to `nla_len'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:384: undefined reference to `nla_next'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:384: undefined reference to `nla_ok'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:385: undefined reference to `nla_type'
/usr/bin/ld: main.o: in function `handle_dm_alert_msg':
/home/mathias/src/dropwatch/src/main.c:404: undefined reference to `nlmsg_data'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:406: undefined reference to `genlmsg_data'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:406: undefined reference to `nla_data'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `handle_dm_alert_msg':
/home/mathias/src/dropwatch/src/main.c:412: undefined reference to `genlmsg_parse'
/usr/bin/ld: main.o: in function `print_nested_port':
/home/mathias/src/dropwatch/src/main.c:443: undefined reference to `nla_parse_nested'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:449: undefined reference to `nla_get_u32'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:453: undefined reference to `nla_get_string'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `handle_dm_packet_alert_msg':
/home/mathias/src/dropwatch/src/main.c:485: undefined reference to `genlmsg_parse'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:490: undefined reference to `nla_get_u64'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:490: undefined reference to `nla_get_string'
/usr/bin/ld: main.o: in function `print_packet_origin':
/home/mathias/src/dropwatch/src/main.c:462: undefined reference to `nla_get_u16'
/usr/bin/ld: main.o: in function `handle_dm_packet_alert_msg':
/home/mathias/src/dropwatch/src/main.c:506: undefined reference to `nla_data'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:507: undefined reference to `nla_len'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:522: undefined reference to `nla_get_u64'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:533: undefined reference to `nla_get_u16'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:536: undefined reference to `nla_len'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:539: undefined reference to `nla_get_u32'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:495: undefined reference to `nla_get_string'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:495: undefined reference to `nla_get_string'
/usr/bin/ld: main.o: in function `print_packet_origin':
/home/mathias/src/dropwatch/src/main.c:462: undefined reference to `nla_get_u16'
/usr/bin/ld: main.o: in function `print_nested_stats':
/home/mathias/src/dropwatch/src/main.c:597: undefined reference to `nla_parse_nested'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:603: undefined reference to `nla_get_u64'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `handle_dm_stats_new_msg':
/home/mathias/src/dropwatch/src/main.c:614: undefined reference to `genlmsg_parse'
/usr/bin/ld: main.o: in function `enable_drop_monitor':
/home/mathias/src/dropwatch/src/main.c:709: undefined reference to `nla_put_flag'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `enable_drop_monitor':
/home/mathias/src/dropwatch/src/main.c:712: undefined reference to `nla_put_flag'
/usr/bin/ld: main.o: in function `disable_drop_monitor':
/home/mathias/src/dropwatch/src/main.c:730: undefined reference to `nla_put_flag'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `disable_drop_monitor':
/home/mathias/src/dropwatch/src/main.c:733: undefined reference to `nla_put_flag'
/usr/bin/ld: main.o: in function `set_alert_mode':
/home/mathias/src/dropwatch/src/main.c:765: undefined reference to `nla_put_u8'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `set_trunc_len':
/home/mathias/src/dropwatch/src/main.c:785: undefined reference to `nla_put_u32'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `set_queue_len':
/home/mathias/src/dropwatch/src/main.c:805: undefined reference to `nla_put_u32'
/usr/bin/ld: main.o: in function `free_netlink_msg':
/home/mathias/src/dropwatch/src/main.c:254: undefined reference to `nlmsg_free'
/usr/bin/ld: main.o: in function `enter_command_line_mode':
/home/mathias/src/dropwatch/src/main.c:865: undefined reference to `readline'
/usr/bin/ld: main.o: in function `send_netlink_message':
/home/mathias/src/dropwatch/src/main.c:265: undefined reference to `nl_send'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:265: undefined reference to `nl_send'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:265: undefined reference to `nl_send'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:265: undefined reference to `nl_send'
/usr/bin/ld: /home/mathias/src/dropwatch/src/main.c:265: undefined reference to `nl_send'
/usr/bin/ld: main.o:/home/mathias/src/dropwatch/src/main.c:265: more undefined references to `nl_send' follow
/usr/bin/ld: main.o: in function `main':
/home/mathias/src/dropwatch/src/main.c:1151: undefined reference to `nl_close'
/usr/bin/ld: lookup_bfd.o: in function `lookup_bfd_init':
/home/mathias/src/dropwatch/src/lookup_bfd.c:48: undefined reference to `bfd_init'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:357: dropwatch] Error 1
make[2]: Leaving directory '/home/mathias/src/dropwatch/src'
make[1]: *** [Makefile:375: all-recursive] Error 1
make[1]: Leaving directory '/home/mathias/src/dropwatch'
make: *** [Makefile:316: all] Error 2

[nhorman]

Thats wierd. Works fine for me.

Based on the output you've provided, it seems like all the c->0 compilations work fine (indicating the headers define prototypes for all of the missing functions), but they can't be located during the final link. That suggests to me that you have object files without any symbols in them.

It would be odd, but you noted that you installed all the requisite -dev packages on your system, but didn't mention anything about the library packages themselves. Does ubuntu have a dependency issue in which the underlying library package isn't auto installed as a depedency? I would start by trying to run: apt-get install libnl-3

to see if you're missing the actual libraries.

[mathiasxx]

I'll poke around, here's a tidbit:

➜  dropwatch git:(master) ✗ dpkg -L libnl-3-200
/.
/etc
/etc/libnl-3
/etc/libnl-3/classid
/etc/libnl-3/pktloc
/lib
/lib/x86_64-linux-gnu
/lib/x86_64-linux-gnu/libnl-3.so.200.26.0
/usr
/usr/share
/usr/share/doc
/usr/share/doc/libnl-3-200
/usr/share/doc/libnl-3-200/README.Debian
/usr/share/doc/libnl-3-200/changelog.Debian.gz
/usr/share/doc/libnl-3-200/copyright
/lib/x86_64-linux-gnu/libnl-3.so.200

[nhorman]

ok, thats a good start, you at least have the libraries available. Try these commands: ls -l /lib/x86_64-linux-gnu/libnl-3.so.200.26.0 objdump -t /lib/x86_64-linux-gnu/libnl-3.so.200.26.0 | grep nlmsg_free

The first command should just tell us that the library is actually there, and not a symlink to some other non-existant file (suggesting that you need another package to resolve the dependency)

The second command should produce some output like this (assuming the first command shows that we have a real file): [nhorman@hmsvengance linux]$ objdump -t /usr/lib64/libnl-3.so.200 | grep nlmsg_free 000000000000ec52 l .text 0000000000000000 .hidden .annobin_nlmsg_free.start 000000000000edd0 l .text 0000000000000000 .hidden .annobin_nlmsg_free.end 000000000000ec60 g F .text 0000000000000170 nlmsg_free

My expectation is that the objdump output will show some output that amounts to something like "no symbols", which suggests ubuntu has done something to mess up the building of the libnl-3 library, preventing it from linking.

[mathiasxx]

Ah ha. yes, I get "no symbols" when running objdump.
Though the so file is there, all 138072 bytes of it.

Thanks for your help. I'll keep poking around.

[nhorman]

you need to open an issue with ubuntu. They've apparently messed up their environment in such a way that you can't link to the libnl library, and thats broken.

[SCadilhac]

Hello, I'm seeing the same issue on Debian 11. This is not a shared library issue - the symbols are present, but in dynamic table:

% objdump --dynamic-syms /lib/x86_64-linux-gnu/libnl-3.so.200 | grep nl_send
0000000000011830 g    DF .text  00000000000000e0  Base        nl_sendto
0000000000011910 g    DF .text  0000000000000182  Base        nl_sendmsg
0000000000012ad0 g    DF .text  000000000000003c  Base        nl_send_sync
0000000000011aa0 g    DF .text  00000000000000da  Base        nl_send_iovec
0000000000012c10 g    DF .text  0000000000000005  Base        nl_send_auto_complete
0000000000011b80 g    DF .text  0000000000000084  Base        nl_send
0000000000011c80 g    DF .text  0000000000000022  Base        nl_send_auto
0000000000011cb0 g    DF .text  0000000000000070  Base        nl_send_simple

The linkage error is rather caused by gcc params order, as suggested/explained here: https://stackoverflow.com/questions/42113237/undefined-reference-when-linking-dynamic-library-in-ubuntu-system

This indeed works (libraries being listed after the object files):

gcc -g -Wall -Werror -I/usr/include/libnl3  -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -g -O2  -o dropwatch main.o lookup.o lookup_kas.o lookup_bfd.o -lnl-3 -lnl-genl-3 -lreadline -lpcap -lbfd

[nhorman]

Ok, so what that article is saying is that ubuntu compiled their library with a linker script to only export symbols in the dynamic table, and built their compiler to only link symbols as needed, which necessitates a specific link order. That still doesn't change the fact that this is ubuntus problem to fix. If you want to work around it modify the src/Makefile.am script such that the AM_LDFLAGS adds a -Wl,--no-as-needed option to override their default compilation (which is likely what they did when they built the package