search

emmericp / MoonGen

MoonGen is a fully scriptable high-speed packet generator built on DPDK and LuaJIT. It can saturate a 10 Gbit/s connection with 64 byte packets on a single CPU core while executing user-provided Lua scripts for each packet. Multi-core support allows for even higher rates. It also features precise and accurate timestamping and rate control.
MIT License
1.04k stars 235 forks source link

Can't build on ARM64 machine with Ubuntu 18.04 #218

Open pavel-odintsov opened 6 years ago

pavel-odintsov commented 6 years ago

Hello!

I have been playing with Arm64 machines for packet processing/packet generation.

I had very nice experience with your tool previously and decided to try it on ARM64 platform.

I use Cavium Thunder X machine with Mellanox 40G CX-4 cards.

Unfortunately, I stuck on build/compilation step:

./build.sh
Build libmoon with MoonGen
==== Building LuaJIT 2.1.0-beta3 ====
make -C src
make[1]: Entering directory '/root/MoonGen/libmoon/deps/luajit/src'
make[1]: Nothing to be done for 'default'.
make[1]: Leaving directory '/root/MoonGen/libmoon/deps/luajit/src'
==== Successfully built LuaJIT 2.1.0-beta3 ====
==== Installing LuaJIT 2.1.0-beta3 to /usr/local ====
mkdir -p /root/MoonGen/libmoon/deps/luajit/usr/local/bin /root/MoonGen/libmoon/deps/luajit/usr/local/lib /root/MoonGen/libmoon/deps/luajit/usr/local/include/luajit-2.1 /root/MoonGen/libmoon/deps/luajit/usr/local/share/man/man1 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/pkgconfig /root/MoonGen/libmoon/deps/luajit/usr/local/share/luajit-2.1.0-beta3/jit /root/MoonGen/libmoon/deps/luajit/usr/local/share/lua/5.1 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/lua/5.1
cd src && install -m 0755 luajit /root/MoonGen/libmoon/deps/luajit/usr/local/bin/luajit-2.1.0-beta3
cd src && test -f libluajit.a && install -m 0644 libluajit.a /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.a || :
rm -f /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2
cd src && test -f libluajit.so && \
  install -m 0755 libluajit.so /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2.1.0 && \
  ldconfig -n /root/MoonGen/libmoon/deps/luajit/usr/local/lib && \
  ln -sf libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so && \
  ln -sf libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2 || :
cd etc && install -m 0644 luajit.1 /root/MoonGen/libmoon/deps/luajit/usr/local/share/man/man1
cd etc && sed -e "s|^prefix=.*|prefix=/usr/local|" -e "s|^multilib=.*|multilib=lib|" luajit.pc > luajit.pc.tmp && \
  install -m 0644 luajit.pc.tmp /root/MoonGen/libmoon/deps/luajit/usr/local/lib/pkgconfig/luajit.pc && \
  rm -f luajit.pc.tmp
cd src && install -m 0644 lua.h lualib.h lauxlib.h luaconf.h lua.hpp luajit.h /root/MoonGen/libmoon/deps/luajit/usr/local/include/luajit-2.1
cd src/jit && install -m 0644 bc.lua bcsave.lua dump.lua p.lua v.lua zone.lua dis_x86.lua dis_x64.lua dis_arm.lua dis_arm64.lua dis_arm64be.lua dis_ppc.lua dis_mips.lua dis_mipsel.lua dis_mips64.lua dis_mips64el.lua vmdef.lua /root/MoonGen/libmoon/deps/luajit/usr/local/share/luajit-2.1.0-beta3/jit
==== Successfully installed LuaJIT 2.1.0-beta3 to /usr/local ====

Note: the development releases deliberately do NOT install a symlink for luajit
You can do this now by running this command (with sudo):

  ln -sf luajit-2.1.0-beta3 /root/MoonGen/libmoon/deps/luajit/usr/local/bin/luajit

Configuration done using x86_64-native-linuxapp-gcc
cc1: error: unknown value ‘corei7’ for -march
== Build lib
cc1: error: unknown value ‘corei7’ for -march
== Build lib/librte_compat
== Build lib/librte_eal
cc1: error: unknown value ‘corei7’ for -march
cc1: error: unknown value ‘corei7’ for -march
== Build lib/librte_eal/common
cc1: error: unknown value ‘corei7’ for -march
== Build lib/librte_eal/linuxapp
cc1: error: unknown value ‘corei7’ for -march
== Build lib/librte_eal/linuxapp/eal
== Build lib/librte_eal/linuxapp/igb_uio
cc1: error: unknown value ‘corei7’ for -march
cc1: error: unknown value ‘corei7’ for -march

  ERROR: Kernel configuration is invalid.
         include/generated/autoconf.h or include/config/auto.conf are missing.
         Run 'make oldconfig && make prepare' on kernel src to fix it.

  WARNING: Symbol version dump ./Module.symvers
           is missing; modules will have no dependencies and modversions.

  CC eal.o
  CC eal_hugepage_info.o
  CC eal_memory.o
gcc: error: unrecognized command line option ‘-m64’
  CC eal_thread.o
gcc: error: unrecognized command line option ‘-m64’
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal.o' failed
make[5]: *** [eal.o] Error 1
make[5]: *** Waiting for unfinished jobs....
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_hugepage_info.o' failed
make[5]: *** [eal_hugepage_info.o] Error 1
  CC eal_log.o
gcc: error: unrecognized command line option ‘-m64’
  CC eal_vfio.o
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_memory.o' failed
make[5]: *** [eal_memory.o] Error 1
gcc: error: unrecognized command line option ‘-m64’
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_thread.o' failed
make[5]: *** [eal_thread.o] Error 1
gcc: error: unrecognized command line option ‘-m64’
gcc: error: unrecognized command line option ‘-m64’
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_log.o' failed
make[5]: *** [eal_log.o] Error 1
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_vfio.o' failed
make[5]: *** [eal_vfio.o] Error 1
/root/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: recipe for target 'eal' failed
make[4]: *** [eal] Error 2
make[4]: *** Waiting for unfinished jobs....
cc1: error: unknown value ‘corei7’ for -march
  Building modules, stage 2.
/root/bpf-next/scripts/Makefile.modpost:43: include/config/auto.conf: No such file or directory
cc1: error: unknown value ‘corei7’ for -march
make[8]: *** No rule to make target 'include/config/auto.conf'.  Stop.
/root/bpf-next/Makefile:1579: recipe for target 'modules' failed
make[7]: *** [modules] Error 2
Makefile:146: recipe for target 'sub-make' failed
make[6]: *** [sub-make] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.module.mk:78: recipe for target 'igb_uio.ko' failed
make[5]: *** [igb_uio.ko] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: recipe for target 'igb_uio' failed
make[4]: *** [igb_uio] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: recipe for target 'linuxapp' failed
make[3]: *** [linuxapp] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: recipe for target 'librte_eal' failed
make[2]: *** [librte_eal] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.sdkbuild.mk:76: recipe for target 'lib' failed
make[1]: *** [lib] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.sdkroot.mk:128: recipe for target 'all' failed
make: *** [all] Error 2
make -C deps/http-parser library
make[1]: Entering directory '/root/MoonGen/libmoon/lua/lib/turbo/deps/http-parser'
gcc  -shared -Wl,-soname=libhttp_parser.so.2.1 -o libhttp_parser.so.2.1 libhttp_parser.o
make[1]: Leaving directory '/root/MoonGen/libmoon/lua/lib/turbo/deps/http-parser'
gcc -Ideps/http-parser/ -shared -O3 -Wall -g  -fPIC deps/http-parser/libhttp_parser.o deps/turbo_ffi_wrap.c -o libtffi_wrap.so  -lcrypto -lssl
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /root/MoonGen/build
Scanning dependencies of target moon
[ 11%] Building C object libmoon/CMakeFiles/moon.dir/src/memory.c.o
[ 11%] Building CXX object libmoon/CMakeFiles/moon.dir/src/main.cpp.o
[ 11%] Building CXX object libmoon/CMakeFiles/moon.dir/src/task.cpp.o
[ 15%] Building C object libmoon/CMakeFiles/moon.dir/src/device.c.o
[ 19%] Building C object libmoon/CMakeFiles/moon.dir/src/i40e.c.o
[ 23%] Building C object libmoon/CMakeFiles/moon.dir/src/util.c.o
[ 26%] Building CXX object libmoon/CMakeFiles/moon.dir/src/lifecycle.cpp.o
[ 30%] Building CXX object libmoon/CMakeFiles/moon.dir/src/barrier.cpp.o
[ 34%] Building CXX object libmoon/CMakeFiles/moon.dir/src/task-results.cpp.o
[ 38%] Building CXX object libmoon/CMakeFiles/moon.dir/src/pipe.cpp.o
[ 42%] Building CXX object libmoon/CMakeFiles/moon.dir/src/lock.cpp.o
[ 46%] Building CXX object libmoon/CMakeFiles/moon.dir/src/namespaces.cpp.o
[ 50%] Building C object libmoon/CMakeFiles/moon.dir/src/ring.c.o
[ 57%] Building C object libmoon/CMakeFiles/moon.dir/src/filter.c.o
[ 57%] Building C object libmoon/CMakeFiles/moon.dir/src/kni.c.o
[ 61%] Building CXX object libmoon/CMakeFiles/moon.dir/src/pcap.cpp.o
[ 65%] Building C object libmoon/CMakeFiles/moon.dir/src/timestamping.c.o
[ 69%] Building C object libmoon/CMakeFiles/moon.dir/src/timestamping_i40e.c.o
c++: error: unrecognized command line option ‘-msse4.2’
cc: error: unrecognized command line option ‘-msse4.2’
[ 73%] Building C object libmoon/CMakeFiles/moon.dir/src/timestamping_ixgbe.c.o
c++: error: unrecognized command line option ‘-msse4.2’
cc: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:62: recipe for target 'libmoon/CMakeFiles/moon.dir/src/main.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/main.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 76%] Building C object libmoon/CMakeFiles/moon.dir/src/timestamping_igb.c.o
cc: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:86: recipe for target 'libmoon/CMakeFiles/moon.dir/src/memory.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/memory.c.o] Error 1
cc: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:110: recipe for target 'libmoon/CMakeFiles/moon.dir/src/task.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/task.cpp.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:134: recipe for target 'libmoon/CMakeFiles/moon.dir/src/device.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/device.c.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:158: recipe for target 'libmoon/CMakeFiles/moon.dir/src/i40e.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/i40e.c.o] Error 1
c++: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:182: recipe for target 'libmoon/CMakeFiles/moon.dir/src/util.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/util.c.o] Error 1
c++: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:206: recipe for target 'libmoon/CMakeFiles/moon.dir/src/lifecycle.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/lifecycle.cpp.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:230: recipe for target 'libmoon/CMakeFiles/moon.dir/src/barrier.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/barrier.cpp.o] Error 1
c++: error: unrecognized command line option ‘-msse4.2’
c++: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:278: recipe for target 'libmoon/CMakeFiles/moon.dir/src/pipe.cpp.o' failed
c++: error: unrecognized command line option ‘-msse4.2’
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/pipe.cpp.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:254: recipe for target 'libmoon/CMakeFiles/moon.dir/src/task-results.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/task-results.cpp.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:302: recipe for target 'libmoon/CMakeFiles/moon.dir/src/lock.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/lock.cpp.o] Error 1
c++: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:326: recipe for target 'libmoon/CMakeFiles/moon.dir/src/namespaces.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/namespaces.cpp.o] Error 1
cc: error: unrecognized command line option ‘-msse4.2’
cc: error: unrecognized command line option ‘-msse4.2’
cc: error: unrecognized command line option ‘-msse4.2’
c++: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:350: recipe for target 'libmoon/CMakeFiles/moon.dir/src/ring.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/ring.c.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:398: recipe for target 'libmoon/CMakeFiles/moon.dir/src/filter.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/filter.c.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:374: recipe for target 'libmoon/CMakeFiles/moon.dir/src/kni.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/kni.c.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:422: recipe for target 'libmoon/CMakeFiles/moon.dir/src/pcap.cpp.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/pcap.cpp.o] Error 1
cc: error: unrecognized command line option ‘-msse4.2’
cc: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:446: recipe for target 'libmoon/CMakeFiles/moon.dir/src/timestamping.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/timestamping.c.o] Error 1
cc: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:470: recipe for target 'libmoon/CMakeFiles/moon.dir/src/timestamping_i40e.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/timestamping_i40e.c.o] Error 1
libmoon/CMakeFiles/moon.dir/build.make:494: recipe for target 'libmoon/CMakeFiles/moon.dir/src/timestamping_ixgbe.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/timestamping_ixgbe.c.o] Error 1
cc: error: unrecognized command line option ‘-msse4.2’
libmoon/CMakeFiles/moon.dir/build.make:518: recipe for target 'libmoon/CMakeFiles/moon.dir/src/timestamping_igb.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/timestamping_igb.c.o] Error 1
CMakeFiles/Makefile2:122: recipe for target 'libmoon/CMakeFiles/moon.dir/all' failed
make[1]: *** [libmoon/CMakeFiles/moon.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Trying to bind interfaces, this will fail if you are not root
Try sudo ./bind-interfaces.sh if this step fails
insmod: ERROR: could not load module ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko: No such file or directory
Traceback (most recent call last):
  File "usertools/dpdk-devbind.py", line 717, in <module>
    main()
  File "usertools/dpdk-devbind.py", line 710, in main
    get_device_details(network_devices)
  File "usertools/dpdk-devbind.py", line 274, in get_device_details
    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
  File "usertools/dpdk-devbind.py", line 149, in check_output
    stderr=stderr).communicate()[0]
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Could not find any inactive interfaces to bind to DPDK. Note that this script does not bind interfaces that are in use by the OS.
Delete IP addresses from interfaces you would like to use with libmoon and run this script again.
You can also use the script dpdk-devbind.py in deps/dpdk/usertools manually to manage interfaces used by libmoon and the OS.

Any options to skip these flags for ARM64 machines? I do not expect great performance but would be nice to have to working.

Thank you!

emmericp commented 6 years ago

I was just thinking about getting an ARM server the other day :)

I think there are currently two problems:

Can you try to:

  1. remove -msse4.2 from both CMakeLists.txt files (total of 4 occurrences I think)
  2. replace all occurences of x86_64 with arm64 in libmoon/build.sh
pavel-odintsov commented 6 years ago

Hello!

Thank you for help!

I applied all fixes and tried two different dpdk build targets:

For first one I received following results:

./build.sh 
Build libmoon with MoonGen
==== Building LuaJIT 2.1.0-beta3 ====
make -C src
make[1]: Entering directory '/root/MoonGen/libmoon/deps/luajit/src'
make[1]: Nothing to be done for 'default'.
make[1]: Leaving directory '/root/MoonGen/libmoon/deps/luajit/src'
==== Successfully built LuaJIT 2.1.0-beta3 ====
==== Installing LuaJIT 2.1.0-beta3 to /usr/local ====
mkdir -p /root/MoonGen/libmoon/deps/luajit/usr/local/bin /root/MoonGen/libmoon/deps/luajit/usr/local/lib /root/MoonGen/libmoon/deps/luajit/usr/local/include/luajit-2.1 /root/MoonGen/libmoon/deps/luajit/usr/local/share/man/man1 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/pkgconfig /root/MoonGen/libmoon/deps/luajit/usr/local/share/luajit-2.1.0-beta3/jit /root/MoonGen/libmoon/deps/luajit/usr/local/share/lua/5.1 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/lua/5.1
cd src && install -m 0755 luajit /root/MoonGen/libmoon/deps/luajit/usr/local/bin/luajit-2.1.0-beta3
cd src && test -f libluajit.a && install -m 0644 libluajit.a /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.a || :
rm -f /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2
cd src && test -f libluajit.so && \
  install -m 0755 libluajit.so /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2.1.0 && \
  ldconfig -n /root/MoonGen/libmoon/deps/luajit/usr/local/lib && \
  ln -sf libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so && \
  ln -sf libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2 || :
cd etc && install -m 0644 luajit.1 /root/MoonGen/libmoon/deps/luajit/usr/local/share/man/man1
cd etc && sed -e "s|^prefix=.*|prefix=/usr/local|" -e "s|^multilib=.*|multilib=lib|" luajit.pc > luajit.pc.tmp && \
  install -m 0644 luajit.pc.tmp /root/MoonGen/libmoon/deps/luajit/usr/local/lib/pkgconfig/luajit.pc && \
  rm -f luajit.pc.tmp
cd src && install -m 0644 lua.h lualib.h lauxlib.h luaconf.h lua.hpp luajit.h /root/MoonGen/libmoon/deps/luajit/usr/local/include/luajit-2.1
cd src/jit && install -m 0644 bc.lua bcsave.lua dump.lua p.lua v.lua zone.lua dis_x86.lua dis_x64.lua dis_arm.lua dis_arm64.lua dis_arm64be.lua dis_ppc.lua dis_mips.lua dis_mipsel.lua dis_mips64.lua dis_mips64el.lua vmdef.lua /root/MoonGen/libmoon/deps/luajit/usr/local/share/luajit-2.1.0-beta3/jit
==== Successfully installed LuaJIT 2.1.0-beta3 to /usr/local ====

Note: the development releases deliberately do NOT install a symlink for luajit
You can do this now by running this command (with sudo):

  ln -sf luajit-2.1.0-beta3 /root/MoonGen/libmoon/deps/luajit/usr/local/bin/luajit

Configuration done using arm64-armv8a-linuxapp-gcc
== Build lib
== Build lib/librte_compat
== Build lib/librte_eal
== Build lib/librte_eal/common
== Build lib/librte_eal/linuxapp
== Build lib/librte_eal/linuxapp/eal
== Build lib/librte_ring
== Build lib/librte_timer
== Build lib/librte_cfgfile
== Build lib/librte_cmdline
== Build lib/librte_kvargs
== Build lib/librte_lpm
== Build lib/librte_acl
== Build lib/librte_jobstats
== Build lib/librte_metrics
== Build lib/librte_power
== Build lib/librte_meter
== Build lib/librte_mempool
== Build lib/librte_eventdev
== Build lib/librte_hash
== Build lib/librte_mbuf
== Build lib/librte_efd
== Build lib/librte_net
== Build lib/librte_cryptodev
== Build lib/librte_reorder
== Build lib/librte_ether
== Build lib/librte_sched
== Build lib/librte_vhost
== Build lib/librte_ip_frag
== Build lib/librte_gro
== Build lib/librte_bitratestats
== Build lib/librte_latencystats
== Build lib/librte_distributor
== Build lib/librte_kni
== Build lib/librte_pdump
== Build lib/librte_port
== Build lib/librte_table
== Build lib/librte_pipeline
== Build buildtools
== Build buildtools/pmdinfogen
== Build drivers
== Build drivers/bus
== Build drivers/mempool
== Build drivers/event
== Build drivers/mempool/ring
== Build drivers/mempool/stack
== Build drivers/event/skeleton
== Build drivers/event/sw
== Build drivers/event/octeontx
== Build drivers/net
== Build drivers/crypto
== Build drivers/crypto/scheduler
== Build drivers/crypto/null
== Build drivers/net/af_packet
== Build drivers/net/ark
== Build drivers/net/bonding
== Build drivers/net/cxgbe
== Build drivers/net/e1000
== Build drivers/net/ena
== Build drivers/net/enic
== Build drivers/net/failsafe
== Build drivers/net/i40e
== Build drivers/net/ixgbe
== Build drivers/net/nfp
== Build drivers/net/liquidio
== Build drivers/net/bnxt
== Build drivers/net/null
== Build drivers/net/qede
== Build drivers/net/ring
== Build drivers/net/tap
== Build drivers/net/thunderx
== Build drivers/net/virtio
== Build drivers/net/vmxnet3
== Build drivers/net/kni
== Build drivers/net/vhost
== Build app
== Build app/proc_info
== Build app/pdump
== Build app/test-crypto-perf
== Build app/test-eventdev
Build complete [arm64-armv8a-linuxapp-gcc]
make -C deps/http-parser library
make[1]: Entering directory '/root/MoonGen/libmoon/lua/lib/turbo/deps/http-parser'
gcc  -shared -Wl,-soname=libhttp_parser.so.2.1 -o libhttp_parser.so.2.1 libhttp_parser.o
make[1]: Leaving directory '/root/MoonGen/libmoon/lua/lib/turbo/deps/http-parser'
gcc -Ideps/http-parser/ -shared -O3 -Wall -g  -fPIC deps/http-parser/libhttp_parser.o deps/turbo_ffi_wrap.c -o libtffi_wrap.so  -lcrypto -lssl
-- Configuring done
-- Generating done
-- Build files have been written to: /root/MoonGen/build
[  3%] Building C object libmoon/CMakeFiles/moon.dir/src/device.c.o
[  7%] Building C object libmoon/CMakeFiles/moon.dir/src/util.c.o
In file included from /root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_byteorder.h:45:0,
                 from /root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_ip.h:83,
                 from /root/MoonGen/libmoon/src/util.c:3:
/root/MoonGen/libmoon/src/util.c: In function ‘rte_rdtsc’:
/root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_common.h:59:13: error: impossible constraint in ‘asm’
 #define asm __asm__
             ^
/root/MoonGen/libmoon/src/util.c:19:2: note: in expansion of macro ‘asm’
  asm volatile("rdtsc" :
  ^~~
libmoon/CMakeFiles/moon.dir/build.make:182: recipe for target 'libmoon/CMakeFiles/moon.dir/src/util.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/util.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_log.h:53:0,
                 from /root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_ethdev.h:178,
                 from /root/MoonGen/libmoon/src/device.c:2:
/root/MoonGen/libmoon/src/rdtsc.h: In function ‘dpdk_receive_with_timestamps_software’:
/root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_common.h:59:13: error: unknown register name ‘%rdx’ in ‘asm’
 #define asm __asm__
             ^
/root/MoonGen/libmoon/src/rdtsc.h:23:2: note: in expansion of macro ‘asm’
  asm volatile("RDTSC\n\t"
  ^~~
/root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_common.h:59:13: error: unknown register name ‘%rcx’ in ‘asm’
 #define asm __asm__
             ^
/root/MoonGen/libmoon/src/rdtsc.h:23:2: note: in expansion of macro ‘asm’
  asm volatile("RDTSC\n\t"
  ^~~
/root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_common.h:59:13: error: unknown register name ‘%rbx’ in ‘asm’
 #define asm __asm__
             ^
/root/MoonGen/libmoon/src/rdtsc.h:23:2: note: in expansion of macro ‘asm’
  asm volatile("RDTSC\n\t"
  ^~~
/root/MoonGen/libmoon/deps/dpdk/arm64-armv8a-linuxapp-gcc/include/rte_common.h:59:13: error: unknown register name ‘%rax’ in ‘asm’
 #define asm __asm__
             ^
/root/MoonGen/libmoon/src/rdtsc.h:23:2: note: in expansion of macro ‘asm’
  asm volatile("RDTSC\n\t"
  ^~~
libmoon/CMakeFiles/moon.dir/build.make:134: recipe for target 'libmoon/CMakeFiles/moon.dir/src/device.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/device.c.o] Error 1
CMakeFiles/Makefile2:122: recipe for target 'libmoon/CMakeFiles/moon.dir/all' failed
make[1]: *** [libmoon/CMakeFiles/moon.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Trying to bind interfaces, this will fail if you are not root
Try sudo ./bind-interfaces.sh if this step fails
insmod: ERROR: could not load module ./arm64-armv8a-linuxapp-gcc/kmod/igb_uio.ko: No such file or directory
Could not find any inactive interfaces to bind to DPDK. Note that this script does not bind interfaces that are in use by the OS.
Delete IP addresses from interfaces you would like to use with libmoon and run this script again.
You can also use the script dpdk-devbind.py in deps/dpdk/usertools manually to manage interfaces used by libmoon and the OS.
root@armlab1:~/MoonGen# find . |grep igb_uio.ko
root@armlab1:~/MoonGen#

Looks like some parts of DPDK still use X86 assembler and gcc on ARM64 does not like it.

For Thunder's target it looks even more strange:

./build.sh 
Build libmoon with MoonGen

==== Building LuaJIT 2.1.0-beta3 ====
make -C src
make[1]: Entering directory '/root/MoonGen/libmoon/deps/luajit/src'

make[1]: Nothing to be done for 'default'.
make[1]: Leaving directory '/root/MoonGen/libmoon/deps/luajit/src'
==== Successfully built LuaJIT 2.1.0-beta3 ====
==== Installing LuaJIT 2.1.0-beta3 to /usr/local ====
mkdir -p /root/MoonGen/libmoon/deps/luajit/usr/local/bin /root/MoonGen/libmoon/deps/luajit/usr/local/lib /root/MoonGen/libmoon/deps/luajit/usr/local/include/luajit-2.1 /root/MoonGen/libmoon/deps/luajit/usr/local/share/man/man1 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/pkgconfig /root/MoonGen/libmoon/deps/luajit/usr/local/share/luajit-2.1.0-beta3/jit /root/MoonGen/libmoon/deps/luajit/usr/local/share/lua/5.1 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/lua/5.1
cd src && install -m 0755 luajit /root/MoonGen/libmoon/deps/luajit/usr/local/bin/luajit-2.1.0-beta3
cd src && test -f libluajit.a && install -m 0644 libluajit.a /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.a || :
rm -f /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2
cd src && test -f libluajit.so && \
  install -m 0755 libluajit.so /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2.1.0 && \
  ldconfig -n /root/MoonGen/libmoon/deps/luajit/usr/local/lib && \
  ln -sf libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so && \
  ln -sf libluajit-5.1.so.2.1.0 /root/MoonGen/libmoon/deps/luajit/usr/local/lib/libluajit-5.1.so.2 || :
cd etc && install -m 0644 luajit.1 /root/MoonGen/libmoon/deps/luajit/usr/local/share/man/man1
cd etc && sed -e "s|^prefix=.*|prefix=/usr/local|" -e "s|^multilib=.*|multilib=lib|" luajit.pc > luajit.pc.tmp && \
  install -m 0644 luajit.pc.tmp /root/MoonGen/libmoon/deps/luajit/usr/local/lib/pkgconfig/luajit.pc && \
  rm -f luajit.pc.tmp
cd src && install -m 0644 lua.h lualib.h lauxlib.h luaconf.h lua.hpp luajit.h /root/MoonGen/libmoon/deps/luajit/usr/local/include/luajit-2.1
cd src/jit && install -m 0644 bc.lua bcsave.lua dump.lua p.lua v.lua zone.lua dis_x86.lua dis_x64.lua dis_arm.lua dis_arm64.lua dis_arm64be.lua dis_ppc.lua dis_mips.lua dis_mipsel.lua dis_mips64.lua dis_mips64el.lua vmdef.lua /root/MoonGen/libmoon/deps/luajit/usr/local/share/luajit-2.1.0-beta3/jit

==== Successfully installed LuaJIT 2.1.0-beta3 to /usr/local ====

Note: the development releases deliberately do NOT install a symlink for luajit
You can do this now by running this command (with sudo):

  ln -sf luajit-2.1.0-beta3 /root/MoonGen/libmoon/deps/luajit/usr/local/bin/luajit

Configuration done using arm64-thunderx-linuxapp-gcc

== Build lib
== Build lib/librte_compat
== Build lib/librte_eal
== Build lib/librte_eal/common
== Build lib/librte_eal/linuxapp
== Build lib/librte_eal/linuxapp/eal
  CC eal.o
  CC eal_hugepage_info.o
  CC eal_pci.o
  CC eal_alarm.o
  CC eal_common_string_fns.o
  CC eal_common_devargs.o
  CC eal_common_options.o
  CC eal_common_thread.o
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_string_fns.c: In function ‘rte_strsplit’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_string_fns.c:69:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_common_string_fns.o' failed
make[5]: *** [eal_common_string_fns.o] Error 1
make[5]: *** Waiting for unfinished jobs....
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal_pci.c: In function ‘pci_scan_one’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal_pci.c:386:1: internal compiler error: Segmentation fault
 }
 ^
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c: In function ‘eal_hugepage_info_init’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:368:1: internal compiler error: Segmentation fault
 }
 ^
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_devargs.c: In function ‘rte_eal_devargs_add’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_devargs.c:189:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_pci.o' failed
make[5]: *** [eal_pci.o] Error 1
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_thread.c: In function ‘eal_thread_dump_affinity’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_thread.c:157:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal_alarm.c: In function ‘rte_eal_alarm_cancel’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal_alarm.c:272:1: internal compiler error: Segmentation fault
 }
 ^
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_hugepage_info.o' failed
make[5]: *** [eal_hugepage_info.o] Error 1
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_common_devargs.o' failed
make[5]: *** [eal_common_devargs.o] Error 1
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_common_thread.o' failed
make[5]: *** [eal_common_thread.o] Error 1
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_alarm.o' failed
make[5]: *** [eal_alarm.o] Error 1
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal.c: In function ‘rte_eal_init’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/linuxapp/eal/eal.c:966:1: internal compiler error: Segmentation fault
 }
 ^
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_options.c: In function ‘eal_parse_set.constprop’:
/root/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_options.c:739:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal.o' failed
make[5]: *** [eal.o] Error 1
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/root/MoonGen/libmoon/deps/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'eal_common_options.o' failed
make[5]: *** [eal_common_options.o] Error 1
/root/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: recipe for target 'eal' failed
make[4]: *** [eal] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: recipe for target 'linuxapp' failed
make[3]: *** [linuxapp] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: recipe for target 'librte_eal' failed
make[2]: *** [librte_eal] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.sdkbuild.mk:76: recipe for target 'lib' failed
make[1]: *** [lib] Error 2
/root/MoonGen/libmoon/deps/dpdk/mk/rte.sdkroot.mk:128: recipe for target 'all' failed
make: *** [all] Error 2
make -C deps/http-parser library
make[1]: Entering directory '/root/MoonGen/libmoon/lua/lib/turbo/deps/http-parser'
gcc  -shared -Wl,-soname=libhttp_parser.so.2.1 -o libhttp_parser.so.2.1 libhttp_parser.o
make[1]: Leaving directory '/root/MoonGen/libmoon/lua/lib/turbo/deps/http-parser'
gcc -Ideps/http-parser/ -shared -O3 -Wall -g  -fPIC deps/http-parser/libhttp_parser.o deps/turbo_ffi_wrap.c -o libtffi_wrap.so  -lcrypto -lssl
-- Configuring done
-- Generating done
-- Build files have been written to: /root/MoonGen/build
[  3%] Building C object libmoon/CMakeFiles/moon.dir/src/memory.c.o
/root/MoonGen/libmoon/src/memory.c:1:10: fatal error: rte_config.h: No such file or directory
 #include <rte_config.h>
          ^~~~~~~~~~~~~~
compilation terminated.
libmoon/CMakeFiles/moon.dir/build.make:86: recipe for target 'libmoon/CMakeFiles/moon.dir/src/memory.c.o' failed
make[2]: *** [libmoon/CMakeFiles/moon.dir/src/memory.c.o] Error 1
CMakeFiles/Makefile2:122: recipe for target 'libmoon/CMakeFiles/moon.dir/all' failed
make[1]: *** [libmoon/CMakeFiles/moon.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Trying to bind interfaces, this will fail if you are not root
Try sudo ./bind-interfaces.sh if this step fails
insmod: ERROR: could not load module ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko: No such file or directory
Could not find any inactive interfaces to bind to DPDK. Note that this script does not bind interfaces that are in use by the OS.
Delete IP addresses from interfaces you would like to use with libmoon and run this script again.
You can also use the script dpdk-devbind.py in deps/dpdk/usertools manually to manage interfaces used by libmoon and the OS.

These segmentation faults look very strange actually.

emmericp commented 6 years ago

Oh right, there is some x86 assembly in libmoon, sorry about that.

I guess I should definitely get one or two of these thunderx machines, they seem nice. I've actually access to one, but I can't use it for DPDK because it's for something completely different.

pavel-odintsov commented 6 years ago

Yes, they're really nice :) I got mine from https://www.worksonarm.com and testing ARM for network workloads.

c3m3gyanesh commented 5 years ago

@pavel-odintsov Did you able to install DPDK on Cavium Thunder X machine ? I am facing a lot of issues with the "internal compiler error: Segmentation fault" too. Please share any info you can. thanks

pavel-odintsov commented 5 years ago

Hello!

Sorry, I do not have Thunder anymore.

On Fri, 26 Oct 2018 at 16:40, Gyanesh Patra notifications@github.com wrote:

@pavel-odintsov https://github.com/pavel-odintsov Did you able to install DPDK on Cavium Thunder X machine ? I am facing a lot of issues with the "internal compiler error: Segmentation fault" too. Please share any info you can. thanks

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/emmericp/MoonGen/issues/218#issuecomment-433451386, or mute the thread https://github.com/notifications/unsubscribe-auth/ACnfZhbxq-SSpuTqpvODTok2WzivmDPzks5uoyz4gaJpZM4Uq1Rx .

-- Sincerely yours, Pavel Odintsov