Can't compile Snabb Switch on Ubuntu 18.04 with ARM64 CPU

[pavel-odintsov]

Hello!

I use Ubuntu 18.04 with 2 x Cavium Thunder X arm64 CPUs with 96 cores. I tried to compile SnabbSwitch and received number of errors about it:

root@armlab1:~# git clone https://github.com/SnabbCo/snabb
Cloning into 'snabb'...
remote: Counting objects: 57238, done.
remote: Total 57238 (delta 0), reused 0 (delta 0), pack-reused 57238
Receiving objects: 100% (57238/57238), 19.73 MiB | 10.18 MiB/s, done.
Resolving deltas: 100% (42560/42560), done.)   
root@armlab1:~# cd snabb
root@armlab1:~/snabb# make -j
make[1]: Entering directory '/root/snabb/lib/luajit'
==== Building LuaJIT 2.1.0-beta2 ====
make -C src
make[2]: Entering directory '/root/snabb/lib/luajit/src'
HOSTCC    host/minilua.o
HOSTCC    host/buildvm_asm.o
HOSTCC    host/buildvm_peobj.o
HOSTCC    host/buildvm_lib.o
HOSTCC    host/buildvm_fold.o
CC        lj_gc.o
CC        lj_char.o
CC        lj_obj.o
CC        lj_buf.o
CC        lj_str.o
CC        lj_tab.o
CC        lj_func.o
CC        lj_udata.o
CC        lj_debug.o
CC        lj_state.o
CC        lj_meta.o
CC        lj_vmevent.o
CC        lj_vmmath.o
CC        lj_strscan.o
CC        lj_strfmt.o
CC        lj_strfmt_num.o
CC        lj_api.o
CC        lj_profile.o
CC        lj_lex.o
CC        lj_parse.o
CC        lj_bcread.o
CC        lj_bcwrite.o
CC        lj_load.o
CC        lj_ir.o
CC        lj_opt_mem.o
CC        lj_opt_narrow.o
CC        lj_opt_dce.o
CC        lj_opt_loop.o
CC        lj_opt_split.o
CC        lj_opt_sink.o
CC        lj_mcode.o
CC        lj_snap.o
CC        lj_asm.o
CC        lj_trace.o
CC        lj_gdbjit.o
CC        lj_ctype.o
CC        lj_cdata.o
CC        lj_cconv.o
CC        lj_ccall.o
CC        lj_ccallback.o
CC        lj_carith.o
CC        lj_clib.o
CC        lj_cparse.o
CC        lj_lib.o
CC        lj_alloc.o
CC        lib_aux.o
CC        lib_package.o
CC        lib_init.o
CC        luajit.o
In file included from lj_asm.c:1595:0:
lj_asm_arm64.h: In function ‘asm_ahustore’:
lj_asm_arm64.h:1085:44: warning: result of ‘4294967282 << 47’ requires 80 bits to represent, but ‘long int’ only has 64 bits [-Wshift-overflow=]
    type = ra_allock(as, (int64_t)LJ_TISNUM << 47, allow);
                                            ^~
In file included from lj_asm.c:1595:0:
lj_asm_arm64.h: In function ‘asm_ahustore’:
lj_asm_arm64.h:1085:44: warning: result of ‘4294967282 << 47’ requires 80 bits to represent, but ‘long int’ only has 64 bits [-Wshift-overflow=]
    type = ra_allock(as, (int64_t)LJ_TISNUM << 47, allow);
                                            ^~
HOSTLINK  host/minilua
host/minilua.o: In function `luaV_concat':
minilua.c:(.text+0x3f3c): undefined reference to `memcpy@GLIBC_2.2.5'
host/minilua.o: In function `luaS_newlstr':
minilua.c:(.text+0x48fc): undefined reference to `memcpy@GLIBC_2.2.5'
host/minilua.o: In function `luaL_addvalue':
minilua.c:(.text+0xfdb4): undefined reference to `memcpy@GLIBC_2.2.5'
collect2: error: ld returned 1 exit status
Makefile:628: recipe for target 'host/minilua' failed
make[2]: *** [host/minilua] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/root/snabb/lib/luajit/src'
Makefile:112: recipe for target 'default' failed
make[1]: *** [default] Error 2
make[1]: Leaving directory '/root/snabb/lib/luajit'
Makefile:11: recipe for target 'all' failed
make: *** [all] Error 2

libc versions on Ubuntu 18.04 machine:

ii  libc-bin                                  2.27-3ubuntu1                              arm64        GNU C Library: Binaries
ii  libc-dev-bin                              2.27-3ubuntu1                              arm64        GNU C Library: Development binaries
ii  libc6:arm64                               2.27-3ubuntu1                              arm64        GNU C Library: Shared libraries
ii  libc6-dev:arm64                           2.27-3ubuntu1                              arm64        GNU C Library: Development Libraries and Header Files

Any suggestions?

Also, I familiar with some very nasty issues with lightuserdata on ARM64: https://github.com/LuaJIT/LuaJIT/issues/49

Briefly: luajit assumes that 47 bits is enough to store pointer but that's not true for ARM64 machines as they use more address space. Probably, it can affect you too and I do not know about any solutions for this issue at this moment.

[eugeneia]

Snabb only supports x86_64 at the moment. I believe the issue you ran into is resolved for GC64 which we will start using with the move to RaptorJIT, but RaptorJIT for now only support x86_64 as well. Note that we also use x86_64 assembly directly in Snabb. So as far as I know, making this fly would require:

• porting RaptorJIT to ARM64 (this might be relatively easy since most (all?) of the required code is already in LuaJIT
• porting any x86_64 specific parts of Snabb to ARM64
• somebody maintaining the result

[pavel-odintsov]

Hello!

Thank you for response! Yes, direct use of assembler can make things complicated :(

Can I ask about benefits of RaptorJIT over LuaJIT?

Thank you!

[eugeneia]

@pavel-odintsov the RaptorJIT landing page says it better than I could, but for me personally the key benefits are:

• fast iteration/experimentation cycle for features that matter to server applications like Snabb
• integration with powerful analysis tooling (Studio)

The general motto, I believe, is narrowing the scope of RaptorJIT relative to LuaJIT on a specific usecase so we can add features we need without getting into the way of LuaJIT’s other downstreams.

[pavel-odintsov]

That's pretty interesting! Thank you for detailed answer! I will consider RaptorJIT as option.