iqiyi / dpvs

DPVS is a high performance Layer-4 load balancer based on DPDK.
Other
3.01k stars 723 forks source link

DPDK 20.11.1 build faild on ubuntu 22.04 #987

Open Xyzo21 opened 1 month ago

Xyzo21 commented 1 month ago

折腾半天,换成了ubuntu 20.04,成功编译。 麻烦注明下吧。。。

ywc689 commented 1 month ago

哪个组件编译报错?具体是什么报错信息?

Xyzo21 commented 1 month ago

哪个组件编译报错?具体是什么报错信息?

在ubuntu22.04,dpvs根目录下执行./scripts/dpdk-build.sh后报错:

[2446/2448] Generating kernel/linux/kni/rte_kni with a custom command
FAILED: kernel/linux/kni/rte_kni.ko 
/usr/bin/make -j4 -C /lib/modules/6.5.0-45-generic/build M=/root/dpvs/dpdk/dpdkbuild/kernel/linux/kni src=/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni 'MODULE_CFLAGS=-include /root/dpvs/dpdk/dpdk-stable-20.11.1/config/rte_config.h -I/root/dpvs/dpdk/dpdk-stable-20.11.1/lib/librte_eal/include -I/root/dpvs/dpdk/dpdk-stable-20.11.1/lib/librte_kni -I/root/dpvs/dpdk/dpdkbuild -I/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni' modules
make: Entering directory '/usr/src/linux-headers-6.5.0-45-generic'
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  You are using:           
  CC [M]  /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_misc.o
/bin/sh: 1: gcc-12: not found
make[2]: *** [scripts/Makefile.build:251: /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_misc.o] Error 127
make[2]: *** Waiting for unfinished jobs....
  CC [M]  /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_net.o
/bin/sh: 1: gcc-12: not found
make[2]: *** [scripts/Makefile.build:251: /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_net.o] Error 127
make[1]: *** [/usr/src/linux-headers-6.5.0-45-generic/Makefile:2039: /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni] Error 2
make: *** [Makefile:234: __sub-make] Error 2
make: Leaving directory '/usr/src/linux-headers-6.5.0-45-generic'
[2447/2448] Compiling C object app/test/dpdk-test.p/test_trace_perf.c.o
ninja: build stopped: subcommand failed.
fail to build dpdk

安装gcc-12后,重新执行./scripts/dpdk-build.sh,报错如下:

FAILED: kernel/linux/kni/rte_kni.ko 
/usr/bin/make -j4 -C /lib/modules/6.5.0-45-generic/build M=/root/dpvs/dpdk/dpdkbuild/kernel/linux/kni src=/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni 'MODULE_CFLAGS=-include /root/dpvs/dpdk/dpdk-stable-20.11.1/config/rte_config.h -I/root/dpvs/dpdk/dpdk-stable-20.11.1/lib/librte_eal/include -I/root/dpvs/dpdk/dpdk-stable-20.11.1/lib/librte_kni -I/root/dpvs/dpdk/dpdkbuild -I/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni' modules
make: Entering directory '/usr/src/linux-headers-6.5.0-45-generic'
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  You are using:           gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  CC [M]  /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_misc.o
  CC [M]  /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_net.o
In file included from /root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:27:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h: In function ‘iova_to_phys’:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h:109:37: error: ‘FOLL_TOUCH’ undeclared (first use in this function); did you mean ‘FOLL_FORCE’?
  109 |                                     FOLL_TOUCH, &page, NULL, NULL);
      |                                     ^~~~~~~~~~
      |                                     FOLL_FORCE
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h:109:37: note: each undeclared identifier is reported only once for each function it appears in
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h:108:15: error: too many arguments to function ‘get_user_pages_remote’
  108 |         ret = get_user_pages_remote(tsk->mm, iova, 1,
      |               ^~~~~~~~~~~~~~~~~~~~~
In file included from ./arch/x86/include/asm/cacheflush.h:5,
                 from ./include/linux/cacheflush.h:5,
                 from ./include/linux/highmem.h:8,
                 from ./include/linux/bvec.h:10,
                 from ./include/linux/skbuff.h:17,
                 from ./include/net/net_namespace.h:43,
                 from ./include/linux/netdevice.h:38,
                 from /root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:14:
./include/linux/mm.h:2397:6: note: declared here
 2397 | long get_user_pages_remote(struct mm_struct *mm,
      |      ^~~~~~~~~~~~~~~~~~~~~
In file included from /root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:22:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h: In function ‘iova_to_phys’:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h:109:37: error: ‘FOLL_TOUCH’ undeclared (first use in this function); did you mean ‘FOLL_FORCE’?
  109 |                                     FOLL_TOUCH, &page, NULL, NULL);
      |                                     ^~~~~~~~~~
      |                                     FOLL_FORCE
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h:109:37: note: each undeclared identifier is reported only once for each function it appears in
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_dev.h:108:15: error: too many arguments to function ‘get_user_pages_remote’
  108 |         ret = get_user_pages_remote(tsk->mm, iova, 1,
      |               ^~~~~~~~~~~~~~~~~~~~~
In file included from ./arch/x86/include/asm/cacheflush.h:5,
                 from ./include/linux/cacheflush.h:5,
                 from ./include/linux/highmem.h:8,
                 from ./include/linux/bvec.h:10,
                 from ./include/linux/skbuff.h:17,
                 from ./include/net/net_namespace.h:43,
                 from ./include/linux/netdevice.h:38,
                 from /root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:9:
./include/linux/mm.h:2397:6: note: declared here
 2397 | long get_user_pages_remote(struct mm_struct *mm,
      |      ^~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/string.h:294,
                 from ./include/linux/bitmap.h:11,
                 from ./include/linux/cpumask.h:12,
                 from ./arch/x86/include/asm/paravirt.h:17,
                 from ./arch/x86/include/asm/cpuid.h:62,
                 from ./arch/x86/include/asm/processor.h:19,
                 from ./arch/x86/include/asm/timex.h:5,
                 from ./include/linux/timex.h:67,
                 from ./include/linux/time32.h:13,
                 from ./include/linux/time.h:60,
                 from ./include/linux/stat.h:19,
                 from ./include/linux/module.h:13,
                 from /root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:7:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c: In function ‘kni_ioctl_create’:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:400:31: warning: passing argument 1 of ‘__builtin_memcpy’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
  400 |                 memcpy(net_dev->dev_addr, dev_info.mac_addr, ETH_ALEN);
      |                        ~~~~~~~^~~~~~~~~~
./include/linux/fortify-string.h:648:27: note: in definition of macro ‘__fortify_memcpy_chk’
  648 |         __underlying_##op(p, q, __fortify_size);                        \
      |                           ^
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:400:17: note: in expansion of macro ‘memcpy’
  400 |                 memcpy(net_dev->dev_addr, dev_info.mac_addr, ETH_ALEN);
      |                 ^~~~~~
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:400:31: note: expected ‘void *’ but argument is of type ‘const unsigned char *’
  400 |                 memcpy(net_dev->dev_addr, dev_info.mac_addr, ETH_ALEN);
      |                        ~~~~~~~^~~~~~~~~~
./include/linux/fortify-string.h:648:27: note: in definition of macro ‘__fortify_memcpy_chk’
  648 |         __underlying_##op(p, q, __fortify_size);                        \
      |                           ^
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:400:17: note: in expansion of macro ‘memcpy’
  400 |                 memcpy(net_dev->dev_addr, dev_info.mac_addr, ETH_ALEN);
      |                 ^~~~~~
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c: In function ‘kni_net_rx_normal’:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:417:17: error: implicit declaration of function ‘netif_rx_ni’; did you mean ‘netif_rx’? [-Werror=implicit-function-declaration]
  417 |                 netif_rx_ni(skb);
      |                 ^~~~~~~~~~~
      |                 netif_rx
In file included from ./include/linux/string.h:294,
                 from ./include/linux/bitmap.h:11,
                 from ./include/linux/cpumask.h:12,
                 from ./arch/x86/include/asm/paravirt.h:17,
                 from ./arch/x86/include/asm/irqflags.h:60,
                 from ./include/linux/irqflags.h:17,
                 from ./include/linux/rcupdate.h:26,
                 from ./include/linux/rculist.h:11,
                 from ./include/linux/pid.h:5,
                 from ./include/linux/sched.h:14,
                 from ./include/linux/ratelimit.h:6,
                 from ./include/linux/dev_printk.h:16,
                 from ./include/linux/device.h:15,
                 from /root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:11:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c: In function ‘kni_net_set_mac’:
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:829:22: warning: passing argument 1 of ‘__builtin_memcpy’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
  829 |         memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
      |                ~~~~~~^~~~~~~~~~
./include/linux/fortify-string.h:648:27: note: in definition of macro ‘__fortify_memcpy_chk’
  648 |         __underlying_##op(p, q, __fortify_size);                        \
      |                           ^
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:829:9: note: in expansion of macro ‘memcpy’
  829 |         memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
      |         ^~~~~~
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:829:22: note: expected ‘void *’ but argument is of type ‘const unsigned char *’
  829 |         memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
      |                ~~~~~~^~~~~~~~~~
./include/linux/fortify-string.h:648:27: note: in definition of macro ‘__fortify_memcpy_chk’
  648 |         __underlying_##op(p, q, __fortify_size);                        \
      |                           ^
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_net.c:829:9: note: in expansion of macro ‘memcpy’
  829 |         memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
      |         ^~~~~~
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:251: /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_net.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/root/dpvs/dpdk/dpdk-stable-20.11.1/kernel/linux/kni/kni_misc.c:406:17: error: implicit declaration of function ‘random_ether_addr’ [-Werror=implicit-function-declaration]
  406 |                 random_ether_addr(net_dev->dev_addr);
      |                 ^~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:251: /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni/kni_misc.o] Error 1
make[1]: *** [/usr/src/linux-headers-6.5.0-45-generic/Makefile:2039: /root/dpvs/dpdk/dpdkbuild/kernel/linux/kni] Error 2
make: *** [Makefile:234: __sub-make] Error 2
make: Leaving directory '/usr/src/linux-headers-6.5.0-45-generic'
ninja: build stopped: subcommand failed.
fail to build dpdk
opsarno commented 4 weeks ago

不得不说 DPVS 文档写的确实差劲,在 EL7 编译也是问题一大堆,例如 pkg-config 要提前编译升级,然后再按文档编译 dpdk & dpvs ,否则 dpdk 编译通过后, dpvs 依赖高版本的 pkg-config,此时再升级,依旧编译不过去的。

再就是连生产环境最佳实践的文档都没有。这玩意儿真有人在生产用吗?