Fast packet routing at Ubuntu 18.04

[JeffryLee]

Hi,

I am trying to use click to build a software router which can achieve Gbps throughput. But I came across some issues.

• My first attempt is to use userlevel click module. I connected three laptops one by one with Gbps Ethernet cables and used the middle one as click router. I did iperf throughput test between end-hosts, the throughput between end-hosts is only around 80 Mbps. But at the same time, the CPU utilization of click is only 25%. It seems that I can still improve the userlevel click to achieve higher throughput. I am wondering whether there is anyway to achieve that??

• At the same time, I also tried to use the click kernel module. But it seems that the kernel module does not compile with Ubuntu 18.04. Specifically, I tried to configure with the following command: ./configure --with-linux=/usr/src/linux-headers-4.18.0-25-generic --with-linux-map=/boot/System.map-4.18.0-25-generic --enable-ipsec --enable-wifi --enable-userlevel --enable-linuxmodule But get an error without any further instruction.

  
  checking for XML_ParserCreateNS in -lexpat... no
  checking for Linux kernel compilation flags...   -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -Iubuntu/include 
  -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -Iubuntu/include  -D__KERNEL__  
  -D__KERNEL__       -fno-strict-aliasing  -fshort-wchar    -fno-PIE -DCC_HAVE_ASM_GOTO -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe  -fno-asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -fno-delete-null-pointer-checks     -O2  --param=allow-store-data-races=0   -fstack-protector-strong   -fno-omit-frame-pointer -fno-optimize-sibling-calls  -fno-var-tracking-assignments -pg  -mfentry -DCC_USING_FENTRY   -fno-strict-overflow -fno-merge-all-constants -fmerge-constants -fno-stack-check -fconserve-stack            $(subdir-ccflags-y) -mrecord-mcount
  -fno-strict-aliasing  -fshort-wchar    -fno-PIE -DCC_HAVE_ASM_GOTO -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe  -fno-asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -fno-delete-null-pointer-checks     -O2  --param=allow-store-data-races=0   -fstack-protector-strong   -fno-omit-frame-pointer -fno-optimize-sibling-calls  -fno-var-tracking-assignments -pg  -mfentry -DCC_USING_FENTRY   -fno-strict-overflow -fno-merge-all-constants -fmerge-constants -fno-stack-check -fconserve-stack           
  configure: making C++-safe versions of Linux kernel headers (may take a while)
  ./include/linux/fs.h: 0),    [__magic_struct_kiocb.ki_filp = filp;
  __magic_struct_kiocb.ki_flags = iocb_flags(filp);
  __magic_struct_kiocb.ki_hint = ki_hint_validate(file_write_hint(filp));
  __magic_struct_kiocb.ki_ioprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_NONE;
  ] (3) at /home/acer2/Documents/click/./linuxmodule/click-linuxtool.pl line 221.
  configure: error: 
  ==============================================

click-linuxtool.pl execution failed.

==============================================

I am wondering whether this is the issue of my configuration or it is because click does not work with the new version of kernel. Thanks!! 

[tbarbette]

Kernel is more or less dead, did not receive patches in years and do not compile with recent kernels.

With DPDK or Netmap, userlevel is faster anyway, mostly because the Linux Kernel stack is overkill for what Click actually uses (raw packets). The PCAP version you probably used incurs super high overhead. https://orbi.uliege.be/bitstream/2268/181954/1/userspaceio.pdf will provide more insight on that topic. With the last versions and 100G hardware, we can forward 80Gbps of real traffic with a single core, so yes you can expect better than your results ;)