OpenFastPath / nginx_ofp

NGiNX integrated with OFP_ODP_DPDK
26 stars 14 forks source link
            README
              ==========
  1. Prerequisites

  2. Preparing build

  3. Configuring NGINX

  4. Running NGINX

  5. Limitations or Known Issues

  6. Changelog

  7. Prerequisites

0.1. Build Prerequisites:

This builds and installs DPDK, ODP-DPDK and OFP.

0.2. Configure DPDK:

ls /sys/devices/system/node/ sudo sh -c 'echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages' sudo sh -c 'echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages'

sudo mkdir /mnt/huge sudo mount -t hugetlbfs nodev /mnt/huge

sudo /sbin/modprobe uio ulimit -Sn 2048

insmod scripts/dpdk/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko

Configure interfaces:

./tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver ============================================

Network devices using kernel driver =================================== 0000:02:00.0 'I350 Gigabit Network Connection' if=p1p1 drv=igb unused=igb_uio *Active* 0000:02:00.1 'I350 Gigabit Network Connection' if=p1p2 drv=igb unused=igb_uio 0000:81:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=eth0 drv=ixgbe unused=igb_uio *Active* 0000:81:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=eth1 drv=ixgbe unused=igb_uio *Active* Other network devices ===================== $ sudo ifconfig eth0 down sudo ./tools/dpdk_nic_bind.py -u 0000:01:00.0 sudo ./tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0 $ ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver ============================================ 0000:81:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused= 0000:81:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused= Network devices using kernel driver =================================== 0000:02:00.0 'I350 Gigabit Network Connection' if=p1p1 drv=igb unused=igb_uio *Active* 0000:02:00.1 'I350 Gigabit Network Connection' if=p1p2 drv=igb unused=igb_uio Other network devices ===================== export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PATH_OFP_ROOT/scripts/check-odp/new-build/lib/ export ODP_PLATFORM_PARAMS="-n 4 -- -p 4" 1. Preparing build ================== set build variables with respective OFP and ODP build path (see 0.1.) export OFP_PATH=$PATH_OFP_ROOT/scripts/check-odp/new-build/ export OFP_ODP=$PATH_OFP_ROOT/scripts/check-odp/new-build/ export RTE_SDK=$PATH_OFP_ROOT/scripts/dpdk/ export RTE_TARGET=x86_64-native-linuxapp-gcc ./configure --sbin-path=/usr/local/nginx_dpdk/nginx --conf-path=/usr/local/nginx_dpdk/nginx.conf --pid-path=/usr/local/nginx_dpdk/nginx.pid --with-http_ssl_module --with-select_module [--without-http_rewrite_module] make install Note: make clean will remove main Makefile and obj dir. So configure shall be run again. It would be useful to save the above build steps in a script and run the script. 2. Configuring NGINX ==================== After build and install, NGINX configuration file will be at /usr/local/nginx_dpdk/nginx.conf. To get NGINX debug logs uncomment the debug line error_log logs/error.log debug To run more on more cores increase the number of worker processes: worker_processes number; Debug logs will be in /usr/local/nginx_dpdk/logs/error.log 3. Running NGINX ================ Be sure you have configured DPDK and interfaces as explained at 0.2. In conf/ofp.conf file, configure the ip address for fp0 interface. To start webserver: sh start_nginx.sh 0 To stop webserver: sh stop_nginx.sh 0 From some other machine try to ping the IP we have set for fp0 interface. If ping works, then open a web browser and try to access the same ip address. If web page loads successfully, webserver with OFP is working. 4. Limitations or Known Issues ============================== a) Current NGINX/OFP implementation assumes interface as 0. It will not work for other interfaces. b) NGINX option tcp_nodelay has no effect. To avoid low request rates, sendfile and tcp_nopush should be enabled, as in the provided nginx.conf file. 5. Changelog ============ Made changes to utilise the OFP event signaling support for handling socket operation like accept and recieve calls on a fd. In ngx_select_module.c file, macro OFP_NOTIFY is enabled to use this implementation. If macro is set to zero, can use older select() implementation.