Not able to build fastbit storage plugin

[kullu208025]

I tried to configure all the hosts(consumer,producer,master and slave) using ansible one by one i.e. by modifying site.yml file all hosts configured successfully except producer it is showing error while configuring ipfixcol

[tomjirsa]

Hi, I have tested deploy on Ubuntu 16.04 and Ubuntu 18.04 and all works.

Please check you have the most recent version of git repository (we added update of ipfixcol install recently).

What distribution do you want to install the Stream4Flow on?

[kullu208025]

Hi, Thanks for your reply I tried it with most recent version of git repository.. my inventory.ini.example file looks like this: [producer] producer ansible_host=196.1.106.216 producer ansible_password=**

[consumer] consumer ansible_host=196.1.106.175 consumer ansible_password=**

[sparkMaster] sparkMaster ansible_host=196.1.106.213 sparkMaster ansible_password=**

[sparkSlave] sparkSlave101 ansible_host=196.1.106.214 sparkSlave101 ansible_password=** sparkSlave102 ansible_host=196.1.106.215 sparkSlave102 ansible_password=**

(all above machines are installed in a virtual box in a server not in my local machine) Now Running command ansible-playbook -i inventory.ini.example site.yml --user spark --ask-su-pass

I get previous error again (in "Build fastbit storage plugin" part) i tried to install ipfixcol directly on that machine (producer) output after command ./configure I get this output below:

`checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether UID '1000' is supported by ustar format... yes checking whether GID '1000' is supported by ustar format... yes checking how to create a ustar tar archive... gnutar checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking how to print strings... printf checking for style of include used by make... GNU checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking dependency style of gcc... none checking for a sed that does not truncate output... /bin/sed checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for fgrep... /bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for ar... ar checking for archiver @FILE support... @ checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc object... ok checking for sysroot... no checking for a working dd... /bin/dd checking how to truncate binary pipes... /bin/dd bs=4096 count=1 checking for mt... mt checking if mt is a manifest tool... no checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... yes checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking for shl_load... no checking for shl_load in -ldld... no checking for dlopen... no checking for dlopen in -ldl... yes checking whether a program can dlopen itself... yes checking whether a statically linked program can dlopen itself... no checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating Makefile config.status: executing depfiles commands config.status: executing libtool commands === configuring in base (/home/spark/ipfixcol/base) configure: running /bin/bash ./configure --disable-option-checking '--prefix=/usr/local' --cache-file=/dev/null --srcdir=. checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether UID '1000' is supported by ustar format... yes checking whether GID '1000' is supported by ustar format... yes checking how to create a ustar tar archive... gnutar checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking how to print strings... printf checking for style of include used by make... GNU checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking dependency style of gcc... gcc3 checking for a sed that does not truncate output... /bin/sed checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for fgrep... /bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for ar... ar checking for archiver @FILE support... @ checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc object... ok checking for sysroot... no checking for a working dd... /bin/dd checking how to truncate binary pipes... /bin/dd bs=4096 count=1 checking for mt... mt checking if mt is a manifest tool... no checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... yes checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking for shl_load... no checking for shl_load in -ldld... no checking for dlopen... no checking for dlopen in -ldl... yes checking whether a program can dlopen itself... yes checking whether a statically linked program can dlopen itself... no checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... no configure: Using username "" and email "" configure: Detected distribution: debian. Run with --with-distro=DISTRO to override checking for flex... flex checking for bison... bison checking for doxygen... doxygen checking for rpmbuild... rpmbuild checking for xsltproc... xsltproc checking for gcc... (cached) gcc checking whether we are using the GNU C compiler... (cached) yes checking whether gcc accepts -g... (cached) yes checking for gcc option to accept ISO C89... (cached) none needed checking whether gcc understands -c and -o together... (cached) yes checking dependency style of gcc... (cached) gcc3 checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking how to run the C++ preprocessor... g++ -E checking for ld used by g++... /usr/bin/ld -m elf_x86_64 checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking for g++ option to produce PIC... -fPIC -DPIC checking if g++ PIC flag -fPIC -DPIC works... yes checking if g++ static flag -static works... yes checking if g++ supports -c -o file.o... yes checking if g++ supports -c -o file.o... (cached) yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... (cached) GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether make sets $(MAKE)... (cached) yes checking whether CC supports -std=gnu++11... yes configure: WARNING: beware of hidden checking for ELF visibility... yes checking for main in -lxml2... yes checking for library containing SSL_library_init... -lssl checking for pkg-config... yes checking for sctp_bindx in -lsctp... yes checking for pthread_create in -lpthread... yes checking for dlopen in -ldl... (cached) yes checking arpa/inet.h usability... yes checking arpa/inet.h presence... yes checking for arpa/inet.h... yes checking fcntl.h usability... yes checking fcntl.h presence... yes checking for fcntl.h... yes checking netdb.h usability... yes checking netdb.h presence... yes checking for netdb.h... yes checking netinet/in.h usability... yes checking netinet/in.h presence... yes checking for netinet/in.h... yes checking stddef.h usability... yes checking stddef.h presence... yes checking for stddef.h... yes checking for stdint.h... (cached) yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking sys/ioctl.h usability... yes checking sys/ioctl.h presence... yes checking for sys/ioctl.h... yes checking sys/socket.h usability... yes checking sys/socket.h presence... yes checking for sys/socket.h... yes checking syslog.h usability... yes checking syslog.h presence... yes checking for syslog.h... yes checking for unistd.h... (cached) yes checking for libxml/xmlversion.h... yes checking netinet/sctp.h usability... yes checking netinet/sctp.h presence... yes checking for netinet/sctp.h... yes checking openssl/ssl.h usability... yes checking openssl/ssl.h presence... yes checking for openssl/ssl.h... yes checking for stdbool.h that conforms to C99... yes checking for _Bool... yes checking for inline... inline checking for int32_t... yes checking for pid_t... yes checking for size_t... yes checking for ssize_t... yes checking for uint16_t... yes checking for uint32_t... yes checking for uint64_t... yes checking for uint8_t... yes checking for malloc... yes checking for realloc... yes checking vfork.h usability... no checking vfork.h presence... no checking for vfork.h... no checking for fork... yes checking for vfork... yes checking for working fork... yes checking for working vfork... (cached) yes checking for localtime_r... yes checking for memset... yes checking for pathconf... yes checking for select... yes checking for socket... yes checking for strchr... yes checking for strdup... yes checking for strerror... yes checking for strrchr... yes checking for strtol... yes checking whether htobe64 is declared... yes checking whether be64toh is declared... yes checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating Makefile config.status: creating ipfixcol.spec config.status: creating ipfixcol.pc config.status: creating src/Makefile config.status: creating src/ipfixviewer/Makefile config.status: creating src/input/tcp/Makefile config.status: creating src/input/udp/Makefile config.status: creating src/input/ipfix/Makefile config.status: creating src/input/sctp/Makefile config.status: creating src/storage/ipfix/Makefile config.status: creating src/storage/dummy/Makefile config.status: creating src/storage/forwarding/Makefile config.status: creating src/intermediate/anonymization/Makefile config.status: creating src/intermediate/dummy/Makefile config.status: creating src/intermediate/joinflows/Makefile config.status: creating src/intermediate/filter/Makefile config.status: creating src/intermediate/odip/Makefile config.status: creating src/intermediate/hooks/Makefile config.status: creating src/intermediate/timenow/Makefile config.status: creating src/utils/Makefile config.status: creating src/utils/ipfixconf/Makefile config.status: creating src/utils/ipfixsend/Makefile config.status: creating src/utils/libsiso/Makefile config.status: creating src/utils/filter/Makefile config.status: creating src/utils/profiles/Makefile config.status: creating src/utils/elements/Makefile config.status: creating src/utils/conversion/Makefile config.status: creating src/utils/template_mapper/Makefile config.status: creating config/Makefile config.status: creating headers/Makefile config.status: creating documentation/doxygen/Makefile config.status: creating documentation/man/Makefile config.status: creating documentation/doxygen/doxyfile config.status: creating pkgconfig.h config.status: pkgconfig.h is unchanged config.status: executing depfiles commands config.status: executing libtool commands

ipfixcol version 0.9.6 Prefix........: /usr/local Distribution..: debian Debug build...: no C Compiler....: gcc -Wall -O2 -std=gnu99 -fvisibility=hidden -I/usr/include/libxml2 -g -O2 -pthread -I/usr/include/libxml2 -DTLS_SUPPORT C++ Compiler..: g++ -std=gnu++11 -g -O2 -I/usr/include/libxml2 Linker........: -ldl -lxml2 rpmbuild......: rpmbuild Build doc.....: yes xsltproc......: xsltproc xsltmanstyle..: /usr/share/xml/docbook/stylesheet/docbook-xsl/manpages/docbook.xsl flex..........: flex bison.........: bison Doxygen.......: doxygen TLS support...: yes SCTP support..: yes

=== configuring in tools/fbitdump (/home/spark/ipfixcol/tools/fbitdump) configure: running /bin/bash ./configure --disable-option-checking '--prefix=/usr/local' --cache-file=/dev/null --srcdir=. checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking how to print strings... printf checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking for a sed that does not truncate output... /bin/sed checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for fgrep... /bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for ar... ar checking for archiver @FILE support... @ checking for strip... strip checking for ranlib... ranlib checking for gawk... no checking for mawk... mawk checking command to parse /usr/bin/nm -B output from gcc object... ok checking for sysroot... no checking for a working dd... /bin/dd checking how to truncate binary pipes... /bin/dd bs=4096 count=1 checking for mt... mt checking if mt is a manifest tool... no checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... yes checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking for shl_load... no checking for shl_load in -ldld... no checking for dlopen... no checking for dlopen in -ldl... yes checking whether a program can dlopen itself... yes checking whether a statically linked program can dlopen itself... no checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... no checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking whether make sets $(MAKE)... yes checking for style of include used by make... GNU checking whether make supports nested variables... yes checking dependency style of gcc... gcc3 configure: Using username "" and email "" configure: Detected distribution: debian. Run with --with-distro=DISTRO to override checking for flex... flex checking for bison... bison checking for rpmbuild... rpmbuild checking for doxygen... doxygen checking for xsltproc... xsltproc checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking how to run the C++ preprocessor... g++ -E checking for ld used by g++... /usr/bin/ld -m elf_x86_64 checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking for g++ option to produce PIC... -fPIC -DPIC checking if g++ PIC flag -fPIC -DPIC works... yes checking if g++ static flag -static works... yes checking if g++ supports -c -o file.o... yes checking if g++ supports -c -o file.o... (cached) yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... (cached) GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking dependency style of g++... gcc3 checking for gawk... (cached) mawk checking whether CC supports -std=gnu++11... yes checking for library containing fastbit_init... no configure: error: Required library fastbit missing configure: error: ./configure failed for tools/fbitdump `

please Help!!!!!

[tomjirsa]

Hi, please send me Linux distribution you are installing S4F on. Ansible for installation is tested for Ubuntu 16.04 and Ubuntu 18.04 and all works.

If you are using virtualbox on a server, I recommend using the provided vagrant installation. It builds all the virtual machines for you - it is tested and it works. If you want another IP addresses of the created machines or more sparkworkers, this can be changed in configuration.yml file.

[kullu208025]

Thanks for your reply. Virtual machines are having ubuntu 16.04 and my local machine is having ubuntu 18.04. And I also have tried using vagrant but faced lots of issues there , So I switched to Ansible. And i think that error is here

[cermmik]

Hello, it is really hard to find out where the problem is because in our environment everything works well. Please check that you have the latest Ansible (in some distributions, new version needs to be installed via pip and not via apt). Based on your log it looks like that the fastbit library is missing. Please ensure that it is correctly installed by libfastbit.yml. You can also try to install IPFXIcol separately (see info at https://github.com/CESNET/ipfixcol).