Build system does not recognize DPDK headers are installed

[jacek99]

I installed DPDK on Fedora 24 directly from the OS repos:

dnf search dpdk
Last metadata expiration check: 3 days, 2:06:20 ago on Fri Jun 24 10:01:55 2016.
================================================================================================ N/S Matched: dpdk ================================================================================================
dpdk.x86_64 : Set of libraries and drivers for fast packet processing
dpdk-doc.noarch : Data Plane Development Kit API documentation
dpdk-devel.x86_64 : Data Plane Development Kit development files
dpdk-tools.x86_64 : Tools for setting up Data Plane Development Kit environment
dpdk-examples.x86_64 : Data Plane Development Kit example applications

sudo dnf install dpdk-devel
Last metadata expiration check: 0:48:50 ago on Mon Jun 27 11:20:35 2016.
Dependencies resolved.
===================================================================================================================================================================================================================
 Package                                             Arch                                            Version                                                 Repository                                       Size
===================================================================================================================================================================================================================
Installing:
 dpdk-devel                                          x86_64                                          2.2.0-5.fc24                                            fedora                                          258 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 258 k
Installed size: 1.4 M
Is this ok [y/N]: y
Downloading Packages:
dpdk-devel-2.2.0-5.fc24.x86_64.rpm                                                                                                                                                 149 kB/s | 258 kB     00:01    
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                               68 kB/s | 258 kB     00:03     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Installing  : dpdk-devel-2.2.0-5.fc24.x86_64                                                                                                                                                                 1/1 
  Verifying   : dpdk-devel-2.2.0-5.fc24.x86_64   

but the build system did not seem to find it:

/configure.py --enable-dpdk
Traceback (most recent call last):
  File "./configure.py", line 421, in <module>
    raise Exception('--enable-dpdk: dpdk/ is empty. Run "git submodule update --init".')
Exception: --enable-dpdk: dpdk/ is empty. Run "git submodule update --init".

[tgrabiec]

Did you run git submodule update --init as the error message suggests?

[jacek99]

I did afterwards and it seemed to work fine. It would be great if it recognized that DPDK dev packages area already installed on the OS (since they are now packaged in Fedora repos and potentially other distros as well), just like it does for all the other system libraries.

[jacek99]

And BTW, that seems to get the latest version of DPDK. Your instructions do not seem to work for that version, some field names have changed (e.g. CONFIG_RTE_MBUF_REFCNT is now CONFIG_RTE_MBUF_REFCNT_ATOMIC) and it failed to build (plus it required clang compiler, which is not documented, so I had to install that one too).

Trying to go back to installing version 1.8 manually as per your wiki page and seeing if that works.

The docs on building with DPDK could probably use some TLC to get them back up to date...

[jacek99]

I was wrong: CONFIG_RTE_MBUF_REFCNT seems to have been removed altogether in the latest DPDK. CONFIG_RTE_MBUF_REFCNT_ATOMIC is a different field.

Should we stick to compiling ONLY with version 1.8.0 or is it OK to try and use newer versions of DPDK?

[jacek99]

sigh it gets worse. Once I got DPDK 1.8.0 it turns out it cannot even be compiled on latest Fedora, since it relies on ancient version of gcc (looks like they switched to clang in latest git version only).

Please advise what is the appropriate thing to do. Cannot build 1.8 on latest Fedora.

================== Installing x86_64-native-linuxapp-gcc
make[5]: Nothing to be done for 'depdirs'.
Configuration done
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
== Build scripts
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
== Build scripts/testhost
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
== Build lib
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
== Build lib/librte_eal
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
== Build lib/librte_eal/common
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
== Build lib/librte_eal/linuxapp
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
== Build lib/librte_eal/linuxapp/igb_uio
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/toolchain/gcc/rte.toolchain-compat.mk:46: You are not using GCC 4.x. This is neither supported, nor tested.
make[8]: *** /lib/modules/4.5.7-300.fc24.x86_64/build: No such file or directory.  Stop.
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.module.mk:79: recipe for target 'igb_uio.ko' failed
make[7]: *** [igb_uio.ko] Error 2
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.subdir.mk:61: recipe for target 'igb_uio' failed
make[6]: *** [igb_uio] Error 2
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.subdir.mk:61: recipe for target 'linuxapp' failed
make[5]: *** [linuxapp] Error 2
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.subdir.mk:61: recipe for target 'librte_eal' failed
make[4]: *** [librte_eal] Error 2
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.sdkbuild.mk:93: recipe for target 'lib' failed
make[3]: *** [lib] Error 2
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.sdkroot.mk:124: recipe for target 'all' failed
make[2]: *** [all] Error 2
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.sdkinstall.mk:58: recipe for target 'x86_64-native-linuxapp-gcc_install' failed
make[1]: *** [x86_64-native-linuxapp-gcc_install] Error 2
/home/jfurmank/Dev/c++/dpdk-1.8.0/mk/rte.sdkroot.mk:102: recipe for target 'install' failed
make: *** [install] Error 2
------------------------------------------------------------------------------
 RTE_TARGET exported as x86_64-native-linuxapp-gcc
------------------------------------------------------------------------------

[vladzcloudius]

@jacek99 The DPDK version we have (as a module) in our seastar library is based on the upstream version 2.1.0 and it should build just fine for all Linux distros we support, which includes latest Fedoras, CentOS and Ubuntu distributions. We also do not support any DPDK version below 2.0.0 because it includes a few features we rely on the older versions do not have.

If you plan to use a seastar/scylladb I'd suggest you use our in-tree DPDK version since it also includes a few important patches the upstream version does not (the last time I checked).