Building wireshark/plugins/btbb on OSX

[1am]

Hi

I've been working on building libbtbb under OSX and I succeeded with the checkout from master branch with a minor modification which makes the building steps look like this:

$ mkdir build
$ cd build
$ wget http://www.tcpdump.org/release/libpcap-1.6.2.tar.gz
$ tar zxvf libpcap-1.6.2.tar.gz
$ cmake -DPCAP_INCLUDE_DIR=`pwd`/libpcap-1.6.2 ..
$ make
$ make install

Encouraged by my probably first ever success in making something with makefiles work i went to the next step from Ubertooth build guide, and that's Wireshark step. After I've figured out where Wireshark 1.12.3 has it's libraries I've downloaded it's sources and adapted the OSX build instructions to look like this:

wget https://1.eu.dl.wireshark.org/src/wireshark-1.12.3.tar.bz2
tar zxvf wireshark-1.12.3.tar.bz2 
mkdir wireshark
cd wireshark
ln -s ../wireshark-1.12.3/packaging/macosx/native-gtk/config.h .
cd ..
cmake -DWIRESHARK_INCLUDE_DIRS="`pwd`;`pwd`/wireshark-1.12.3" -DWIRESHARK_LIBRARIES=/Applications/Wireshark.app/Contents/Resources/lib/ ..

Output from cmake looks like this (I've added one message for debugging):

$ cmake -DWIRESHARK_INCLUDE_DIRS="`pwd`;`pwd`/wireshark-1.12.3" -DWIRESHARK_LIBRARIES=/Applications/Wireshark.app/Contents/Resources/lib/ ..
-- The C compiler identification is AppleClang 6.0.0.6000054
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Plugin will be installed in: ~/.wireshark/plugins
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.28") 
-- checking for one of the modules 'glib-2.0>=2.4.0'
-- Found GLib2: glib-2.0;intl /usr/local/Cellar/glib/2.42.0/include/glib-2.0;/usr/local/Cellar/glib/2.42.0/lib/glib-2.0/include;/usr/local/opt/gettext/include
-- Looking for include file glib/gregex.h
-- Looking for include file glib/gregex.h - not found
-- Looking for include file glib/gchecksum.h
-- Looking for include file glib/gchecksum.h - not found
-- WIRESHARK_INCLUDE_DIRS = /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build;/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build

Now calling make is the place I've got stuck at. For the most of the time I've had problems with inclusion paths but after this got resolved (with the nasty local wireshark directory + symlink to config.h) but after that I've started getting errors which look more like the btbb plugin is supposed to be built against different version of Wireshark headers:

$ make
Scanning dependencies of target btbb
[ 25%] Building C object CMakeFiles/btbb.dir/plugin.c.o
[ 50%] Building C object CMakeFiles/btbb.dir/packet-btbb.c.o
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:32:
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:41:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/include/varargs.h:25:4: error: "Please use <stdarg.h> instead of <varargs.h>"
  #error "Please use <stdarg.h> instead of <varargs.h>"
   ^
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:32:
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:44:
In file included from /usr/local/Cellar/glib/2.42.0/include/glib-2.0/glib.h:30:
In file included from /usr/local/Cellar/glib/2.42.0/include/glib-2.0/glib/galloca.h:32:
In file included from /usr/local/Cellar/glib/2.42.0/include/glib-2.0/glib/gtypes.h:32:
/usr/local/Cellar/glib/2.42.0/lib/glib-2.0/include/glibconfig.h:103:9: warning: 'GLIB_MINOR_VERSION' macro redefined
#define GLIB_MINOR_VERSION 42
        ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark/config.h:50:9: note: previous definition is here
#define GLIB_MINOR_VERSION 12
        ^
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:32:
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:44:
In file included from /usr/local/Cellar/glib/2.42.0/include/glib-2.0/glib.h:30:
In file included from /usr/local/Cellar/glib/2.42.0/include/glib-2.0/glib/galloca.h:32:
In file included from /usr/local/Cellar/glib/2.42.0/include/glib-2.0/glib/gtypes.h:32:
/usr/local/Cellar/glib/2.42.0/lib/glib-2.0/include/glibconfig.h:104:9: warning: 'GLIB_MICRO_VERSION' macro redefined
#define GLIB_MICRO_VERSION 0
        ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark/config.h:47:9: note: previous definition is here
#define GLIB_MICRO_VERSION 9
        ^
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:32:
In file included from /Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:113:15: error: unknown type name 'WS_MSVC_NORETURN'
WS_DLL_PUBLIC WS_MSVC_NORETURN void proto_report_dissector_bug(const char *message) G_GNUC_NORETURN;
              ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:113:32: error: expected identifier or '('
WS_DLL_PUBLIC WS_MSVC_NORETURN void proto_report_dissector_bug(const char *message) G_GNUC_NORETURN;
                               ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:158:2: warning: implicit declaration of function 'proto_report_dissector_bug' is invalid in C99 [-Wimplicit-function-declaration]
        DISSECTOR_ASSERT(tvb_length_remaining(tvb, offset) >= 1);
        ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:142:4: note: expanded from macro 'DISSECTOR_ASSERT'
   __DISSECTOR_ASSERT (expression, __FILE__, __LINE__))) \
   ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:180:4: note: expanded from macro '__DISSECTOR_ASSERT'
  (REPORT_DISSECTOR_BUG( \
   ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:116:2: note: expanded from macro 'REPORT_DISSECTOR_BUG'
        proto_report_dissector_bug(message)
        ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:277:78: error: expected ')'
dissect_btbb(tvbuff_t *tvb, packet_info *pinfo, proto_tree *tree, void *data _U_)
                                                                             ^
/Users/pb/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:277:13: note: to match this '('
dissect_btbb(tvbuff_t *tvb, packet_info *pinfo, proto_tree *tree, void *data _U_)
            ^
3 warnings and 4 errors generated.
make[2]: *** [CMakeFiles/btbb.dir/packet-btbb.c.o] Error 1
make[1]: *** [CMakeFiles/btbb.dir/all] Error 2
make: *** [all] Error 2

I've tried different sources, even ones from apt-get repository (after unpacking with ar -x libwireshark-dev_1.8.2-5wheezy14_ia64.deb, there's a tar zxvf data.tar.gz file which contains ./usr/include/wireshark files which are needed). None of the approaches work but I can't really understand what am I doing wrong?

[1am]

I've a few more attempts to compile the plugin, and got to the point where compilation goes through and linker fails. To get there, after looking through wireshark's sources, I've had to add the following flags to CMakeLists.txt

add_definitions(-DHAVE_STDARG_H)
add_definitions(-DWS_MSVC_NORETURN=/**/)
add_definitions(-D_U_=/**/)

• HAVE_STDARG_H solves the #error "Please use <stdarg.h> instead of <varargs.h>" problem with Wireshark
• WS_MSVC_NORETURN - I've found a solution like this somewhere on the web. Not really sure why they are used for and what library/header they come from...
• U - This overrides compiler failing on void *data U argument in libbtbb/wireshark.plugins/btbb/packet-btbb.c

So I've got to the following steps for building the btbb wireshark plugin (after adding flags to CMakeLists.txt)

mkdir build
cd build
wget https://1.eu.dl.wireshark.org/src/wireshark-1.12.3.tar.bz2
tar zxvf wireshark-1.12.3.tar.bz2 
mkdir wireshark
cd wireshark
ln -s ../wireshark-1.12.3/packaging/macosx/native-gtk/config.h .
cd ..
cmake -DWIRESHARK_INCLUDE_DIRS="`pwd`;`pwd`/wireshark-1.12.3" -DWIRESHARK_LIBRARIES=/Applications/Wireshark.app/Contents/Resources/lib/ ..
make

This results in following output: cmake

$ cmake -DWIRESHARK_INCLUDE_DIRS="`pwd`;`pwd`/wireshark-1.12.3" -DWIRESHARK_LIBRARIES=/Applications/Wireshark.app/Contents/Resources/lib/  ..
-- The C compiler identification is AppleClang 6.0.0.6000056
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Plugin will be installed in: ~/.wireshark/plugins
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.28") 
-- checking for one of the modules 'glib-2.0>=2.4.0'
-- Found GLib2: glib-2.0;intl /usr/local/Cellar/glib/2.42.1/include/glib-2.0;/usr/local/Cellar/glib/2.42.1/lib/glib-2.0/include;/usr/local/opt/gettext/include
-- Looking for include file glib/gregex.h
-- Looking for include file glib/gregex.h - not found
-- Looking for include file glib/gchecksum.h
-- Looking for include file glib/gchecksum.h - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build

make

$ make
Scanning dependencies of target btbb
[ 25%] Building C object CMakeFiles/btbb.dir/plugin.c.o
[ 50%] Building C object CMakeFiles/btbb.dir/packet-btbb.c.o
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:32:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:44:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib.h:30:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/galloca.h:32:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/gtypes.h:32:
/usr/local/Cellar/glib/2.42.1/lib/glib-2.0/include/glibconfig.h:103:9: warning: 'GLIB_MINOR_VERSION' macro redefined
#define GLIB_MINOR_VERSION 42
        ^
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark/config.h:50:9: note: previous definition is here
#define GLIB_MINOR_VERSION 12
        ^
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:32:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:44:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib.h:30:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/galloca.h:32:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/gtypes.h:32:
/usr/local/Cellar/glib/2.42.1/lib/glib-2.0/include/glibconfig.h:104:9: warning: 'GLIB_MICRO_VERSION' macro redefined
#define GLIB_MICRO_VERSION 1
        ^
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark/config.h:47:9: note: previous definition is here
#define GLIB_MICRO_VERSION 9
        ^
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btbb.c:32:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:46:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/emem.h:29:
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/ws_symbol_export.h:142:9: warning: 
      'WS_DLL_PUBLIC' macro redefined
#define WS_DLL_PUBLIC   WS_DLL_PUBLIC_DEF extern
        ^
<command line>:2:9: note: previous definition is here
#define WS_DLL_PUBLIC /**/
        ^
3 warnings generated.
[ 75%] Building C object CMakeFiles/btbb.dir/packet-btlmp.c.o
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btlmp.c:31:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:44:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib.h:30:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/galloca.h:32:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/gtypes.h:32:
/usr/local/Cellar/glib/2.42.1/lib/glib-2.0/include/glibconfig.h:103:9: warning: 'GLIB_MINOR_VERSION' macro redefined
#define GLIB_MINOR_VERSION 42
        ^
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark/config.h:50:9: note: previous definition is here
#define GLIB_MINOR_VERSION 12
        ^
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btlmp.c:31:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:44:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib.h:30:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/galloca.h:32:
In file included from /usr/local/Cellar/glib/2.42.1/include/glib-2.0/glib/gtypes.h:32:
/usr/local/Cellar/glib/2.42.1/lib/glib-2.0/include/glibconfig.h:104:9: warning: 'GLIB_MICRO_VERSION' macro redefined
#define GLIB_MICRO_VERSION 1
        ^
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark/config.h:47:9: note: previous definition is here
#define GLIB_MICRO_VERSION 9
        ^
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btlmp.c:31:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/packet.h:26:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/proto.h:46:
In file included from /Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/epan/emem.h:29:
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/build/wireshark-1.12.3/ws_symbol_export.h:142:9: warning: 
      'WS_DLL_PUBLIC' macro redefined
#define WS_DLL_PUBLIC   WS_DLL_PUBLIC_DEF extern
        ^
<command line>:2:9: note: previous definition is here
#define WS_DLL_PUBLIC /**/
        ^
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btlmp.c:2262:31: warning: implicit conversion from 'long' to
      'guint32' (aka 'unsigned int') changes value from 281474976710655 to 4294967295 [-Wconstant-conversion]
                        FT_UINT64, BASE_HEX, NULL, 0x0000ffffffffffff,
                                                   ^~~~~~~~~~~~~~~~~~
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btlmp.c:2267:31: warning: implicit conversion from 'long' to
      'guint32' (aka 'unsigned int') changes value from 281474976710655 to 4294967295 [-Wconstant-conversion]
                        FT_UINT64, BASE_HEX, NULL, 0x0000ffffffffffff,
                                                   ^~~~~~~~~~~~~~~~~~
/Users/pb/Documents/tmp/ubertooth/libbtbb/wireshark/plugins/btbb/packet-btlmp.c:2272:31: warning: implicit conversion from 'long' to
      'guint32' (aka 'unsigned int') changes value from 281474976710655 to 4294967295 [-Wconstant-conversion]
                        FT_UINT64, BASE_HEX, NULL, 0x0000ffffffffffff,
                                                   ^~~~~~~~~~~~~~~~~~
6 warnings generated.
Linking C shared module btbb.so
ld: library not found for -lwireshark
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [btbb.so] Error 1
make[1]: *** [CMakeFiles/btbb.dir/all] Error 2
make: *** [all] Error 2

Given all that here are my questions regarding OSX build: 1) Where could I get the wireshark library from? I've got the following libraries in my wireshark's lib folder:

libwireshark.5.0.3.dylib -> libwireshark.5.dylib
libwireshark.5.dylib
libwireshark.dylib -> libwireshark.5.dylib

How could I make the build scripts see the library? I've also tried building wireshark from sources but it has some problems with GTK+3 which I can't seem to install correctly via homebrew. aybe I could maybe get the lib from source build? 2) Why is the U flag in libbtbb/wireshark.plugins/btbb/packet-btbb.c ? 3) What else could be broken after overriding the flags which I've added to CMakeLists.txt? 4) Is there a remote chance that I'm ever going to succeed with building the btbb plugin for Wireshark for OSX? :)

[miminashi]

I got similar error.

it looks ok.

$ cmake -DWIRESHARK_INCLUDE_DIRS="/usr/local/Cellar/wireshark/1.12.5/include;/usr/local/Cellar/wireshark/1.12.5/include/wireshark" -DWIRESHARK_LIBRARIES="/usr/local/Cellar/wireshark/1.12.5" ..
-- Plugin will be installed in: ~/.wireshark/plugins
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/miminashi/projects/libbtbb/wireshark/plugins/btle/build

but, I also got ld: library not found for -lwireshark

$ make
[ 33%] Generating plugin.c
Scanning dependencies of target btle
[ 66%] Building C object CMakeFiles/btle.dir/plugin.c.o
[100%] Building C object CMakeFiles/btle.dir/packet-btle.c.o
/Users/miminashi/projects/libbtbb/wireshark/plugins/btle/packet-btle.c:301:6: warning: implicit declaration of function 'check_col' is invalid in C99
      [-Wimplicit-function-declaration]
        if (check_col(pinfo->cinfo, COL_PROTOCOL))
            ^
1 warning generated.
Linking C shared module btle.so
ld: library not found for -lwireshark
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [btle.so] Error 1
make[1]: *** [CMakeFiles/btle.dir/all] Error 2
make: *** [all] Error 2

some idea?