[Feature Request] ARM Support

[palexster]

Is your feature request related to a problem? Please describe.

Polycube should support ARM and ARM64 architectures

Describe the solution you'd like

Some dependencies are not available in ARM64, following the installation script. Moreover, when launched the Polycube daemon is not behaving correctly

Additional context

Tested on Raspberry PI4 with 4GB of Ram

[michelsciortino]

The missing dependency is the arm64 version of libyang so it needs to be compiled locally on the arm environment.

But sadly this is not enough: launching the daemon it does not respond and uses 100% of one CPU core.

By debugging with GDB via terminal, it is possible to see that the daemon enters in an infinite loop in this line: https://github.com/polycube-network/polycube/blob/40a49dfb40bcc223cd77f71545e8155bccbe82bf/src/polycubed/src/config.cpp#L496 The ch variable is assigned to -1 which indicates an EOF; this is an expected behavior but in the comparison with -1 it is seen as an unsigned int so it becomes 255.

As this post suggests: https://stackoverflow.com/questions/17070958/c-why-does-getopt-return-255-on-linux?answertab=votes#tab-top

compiling the source code with the -fsigned-char GCC flag, the char variables should be seen as signed int so the code shouldn't lock on this infinite loop.

[michelsciortino]

By inserting

set(GCC_SIGNED_CHAR_FLAG "-fsigned-char")
add_definitions(${GCC_SIGNED_CHAR_FLAG})

in the main CMakeList file, the code starts working but a new problem comes out:

[mrbluecoat]

Here's my install process and like @michelsciortino launching the daemon or rebooting causes it to not respond and uses 100% of one CPU core

DEBIAN_FRONTEND=noninteractive apt -t buster-backports install -yq golang-go git build-essential cmake bison flex libelf-dev libllvm8 llvm-8-dev libclang-8-dev libpcap-dev libnl-route-3-dev libnl-genl-3-dev uuid-dev pkg-config autoconf libtool m4 automake libssl-dev kmod jq bash-completion gnupg2 libfl-dev libnetfilter-conntrack-dev libyang-dev

MAKECORES=$(($(getconf _NPROCESSORS_ONLN) - 2))
MAKECORES=$(test ${MAKECORES} -gt 0 && echo ${MAKECORES} || echo 1)

git clone https://github.com/oktal/pistache.git
cd pistache
git checkout 117db02eda9d63935193ad98be813987f6c32b33
git submodule update --init
mkdir build && cd build
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DPISTACHE_USE_SSL=ON ..
#make -j $(getconf _NPROCESSORS_ONLN)
make -j ${MAKECORES}
make install
cd ../..

git clone --branch v3.5 https://github.com/mfontanini/libtins.git
cd libtins
mkdir build && cd build
cmake -DLIBTINS_ENABLE_CXX11=ON -DLIBTINS_BUILD_EXAMPLES=OFF -DLIBTINS_BUILD_TESTS=OFF -DLIBTINS_ENABLE_DOT11=OFF -DLIBTINS_ENABLE_PCAP=OFF -DLIBTINS_ENABLE_WPA2=OFF -DLIBTINS_ENABLE_WPA2_CALLBACKS=OFF ..
#make -j $(getconf _NPROCESSORS_ONLN)
make -j ${MAKECORES}
make install
ldconfig
cd ../..

test -z "${GOPATH}" && mkdir -p $HOME/go && export GOPATH=$HOME/go

git clone https://github.com/polycube-network/polycube
cd polycube
git submodule update --init --recursive
mkdir build && cd build
cmake .. -DENABLE_PCN_IPTABLES=ON -DENABLE_SERVICE_BRIDGE=OFF -DENABLE_SERVICE_DDOSMITIGATOR=OFF -DENABLE_SERVICE_FIREWALL=OFF -DENABLE_SERVICE_HELLOWORLD=OFF -DENABLE_SERVICE_IPTABLES=ON -DENABLE_SERVICE_K8SFILTER=OFF -DENABLE_SERVICE_K8SWITCH=OFF -DENABLE_SERVICE_LBDSR=OFF -DENABLE_SERVICE_LBRP=OFF -DENABLE_SERVICE_NAT=OFF -DENABLE_SERVICE_PBFORWARDER=OFF -DENABLE_SERVICE_ROUTER=OFF -DENABLE_SERVICE_SIMPLEBRIDGE=OFF -DENABLE_SERVICE_SIMPLEFORWARDER=OFF -DENABLE_SERVICE_TRANSPARENTHELLOWORLD=OFF -DENABLE_SERVICE_SYNFLOOD=OFF -DENABLE_SERVICE_PACKETCAPTURE=OFF
#make -j $(getconf _NPROCESSORS_ONLN)
make -j ${MAKECORES}
make install
cd ../..

systemctl start polycubed && systemctl enable polycubed

# check service status
systemctl status polycubed

# This mode comes with performance gain, especially when policy are configured to DROP traffic -- operates only on interfaces that support XDP native mode
#pcn-iptables-init-xdp

pcn-iptables-init

[mrbluecoat]

P.S. I've also tried it without git checkout 117db02eda9d63935193ad98be813987f6c32b33 and --branch v3.5 but same behavior.

[michelsciortino]

Some months are passed since I've put hands on this but I remember the execution stops here:

https://github.com/polycube-network/polycube/blob/1bb4fd9ad79e8fd3d592674933612d58cd8a2450/src/polycubed/src/config.cpp#L496

getopt_long returns EOF which gets converted into 255, so the while keeps looping forever.

[mrbluecoat]

Thanks @michelsciortino! I added your fix from above

echo 'set(GCC_SIGNED_CHAR_FLAG "-fsigned-char")' >> ./CMakeLists.txt
echo 'add_definitions(${GCC_SIGNED_CHAR_FLAG})' >> ./CMakeLists.txt

But it didn't seem to make a difference in my testing -- same 100% CPU issue and this error:

# pcn-iptables-init
starting pcn-iptables ...
Post "http://localhost:9000/polycube/v1/iptables/pcn-iptables/": dial tcp 127.0.0.1:9000: connect: connection refused

Looking at the compile logs, here are some entries that may be helpful:

iptables.c: In function ‘show_pcn_rules’:
iptables.c:285:13: warning: comparison between pointer and zero character constant [-Wpointer-compare]
   if (chain != '\0'){
             ^~
iptables.c:285:7: note: did you mean to dereference the pointer?
   if (chain != '\0'){
       ^
iptables.c:291:13: warning: comparison between pointer and zero character constant [-Wpointer-compare]
   if (chain != '\0'){
             ^~
iptables.c:291:7: note: did you mean to dereference the pointer?
   if (chain != '\0'){
       ^

libxt_comment.c: In function ‘comment_xlate’:
libxt_comment.c:62:46: warning: ‘"’ directive output may be truncated writing 1 byte into a region of size between 0 and 255 [-Wformat-truncation=]
   snprintf(comment, XT_MAX_COMMENT_LEN, "\"%s\"",
                                              ^~
libxt_comment.c:62:3: note: ‘snprintf’ output between 3 and 258 bytes into a destination of size 256
   snprintf(comment, XT_MAX_COMMENT_LEN, "\"%s\"",
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     commentinfo->comment);
     ~~~~~~~~~~~~~~~~~~~~~
libxt_comment.c:59:46: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 254 [-Wformat-truncation=]
   snprintf(comment, XT_MAX_COMMENT_LEN, "\\\"%s\\\"",
                                              ^~
libxt_comment.c:59:3: note: ‘snprintf’ output between 5 and 260 bytes into a destination of size 256
   snprintf(comment, XT_MAX_COMMENT_LEN, "\\\"%s\\\"",
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     commentinfo->comment);
     ~~~~~~~~~~~~~~~~~~~~~

/root/polycube/src/libs/viface/viface.cpp:402:39: warning: throw will always call terminate() [-Wterminate]
         throw runtime_error(what.str());
                                       ^
/root/polycube/src/libs/viface/viface.cpp:402:39: note: in C++11 destructors default to noexcept

/root/polycube/src/libs/bcc/src/cc/bcc_exception.h:24:7: warning: ‘__stp.ebpf::StatusTuple::code_’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 class StatusTuple {
       ^~~~~~~~~~~

parser.yy:19.1-28: warning: deprecated directive, use ‘%define api.namespace {ebpf::cc}’ [-Wdeprecated]
 %define namespace "ebpf::cc"
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

/usr/lib/llvm-8/include/clang/AST/DeclOpenMP.h:98:1: warning: multi-line comment [-Wcomment]
 /// #pragma omp declare reduction (foo : int,float : omp_out += omp_in) \
 ^

usr/lib/llvm-8/include/clang/AST/Attr.h:263:17: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
     ParamIdx P(*reinterpret_cast<ParamIdx *>(&S));
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/root/polycube/src/polycubed/src/server/Types/Scalar.cpp:47:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^

[mrbluecoat]

FYI, while we wait for an ARM version I found https://github.com/sematext/oxdpus worked for my needs in the interim in case it helps anyone.

[gnu300]

I just confirmed that this is still an issue, since mostly all SmartNICs are based on ARM the deployment area of polycube is limited. Does anybody have an idea what the root cause is?

[sebymiano]

@gnu300 Are BCC samples working in your SmartNICs? Polycube relies on BCC, than a first step to make it work would be to make sure that BCC works as well.

In addition, TTBOMK current SmartNICs do not support tail calls, which Polycube uses to create the different NF chains. If you really want to run Polycube on the SmartNIC, this will require some internal changes.

[gnu300]

Thanks for your quick reply, well we are getting our SmartNICs (https://www.mellanox.com/files/doc-2020/pb-connectx-6-dx-en-card.pdf) by the end of this week, therefore i can't check if the BCC samples are working. I didn't find any hint in the documentation etc. that the SmartNICs do not support tail calls, do you have any reference for that? Thanks!

[sebymiano]

My bad. I thought that you were using a SmartNIC with native eBPF offloading (e.g., Netronome or FPGA-based). For that SmartNIC it should be possible to perform tail calls; at the end, it is just running a linux kernel with eBPF support on the ARM processor (which has also its eBPF-JIT). I guess it would be better to first check if BCC is working. It should, as I can see here (https://github.com/iovisor/bcc/pull/998). Then, go back and try is Polycube is working as well.

This issue was created within an old version of Polycube (with an old BCC), then the problem may be gone.