Issue using XDP_PASS

[jeff-2]

Hi, I have been testing out the recently added XDP support but have had issues trying to use XDP_PASS. Skimming over the driver code it seems like it should be supported, and perhaps the issue is in my configuration. I couldn't find any ENA specific examples/recommendations for XDP configuration, so maybe I'm missing something. This is my test example:

Steps to reproduce:

• Boot ec2 instance with ENA support on Amazon Linux 2 AMI (Amazon Linux 2 AMI 2.0.20200406.0 x86_64 HVM gp2 - amzn2-ami-hvm-2.0.20200406.0-x86_64-gp2 (ami-0323c3dd2da7fb37d))
• amazon-linux-extras install kernel-ng
• Reboot instance

# Install dependencies
yum install -y elfutils-libelf-devel perf kernel-headers clang llvm flex bison git
cd /mnt/
git clone https://github.com/the-tcpdump-group/libpcap
cd libpcap
./configure
make
make install

# Build xdp tutorial samples
cd /mnt/
git clone --recursive https://github.com/xdp-project/xdp-tutorial
cd xdp-tutorial
make

# Configure settings
ip link set dev eth0 mtu 1500
ethtool -L eth0 combined 4

# Install basic XDP_PASS program in native mode
cd basic01-xdp-pass
./xdp_pass_user -N --dev eth0

Expected result: All packets are passed up to normal network stack for processing Actual result: Connectivity to instance is lost, instance reachability check fails

I have had success loading an XDP program and using XDP_TX. Is XDP_PASS indeed a supported action, and if so can you provide any recommendations for configuring XDP on ec2 instances?

$ modinfo ena
filename:       /lib/modules/5.4.20-12.75.amzn2.x86_64/kernel/drivers/amazon/net/ena/ena.ko
version:        2.2.3g
license:        GPL
description:    Elastic Network Adapter (ENA)
author:         Amazon.com, Inc. or its affiliates
srcversion:     BB8E728EB906AFCCDAAF8EB
alias:          pci:v00001D0Fd0000EC21sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd0000EC20sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd00001EC2sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd00000EC2sv*sd*bc*sc*i*
depends:        
retpoline:      Y
intree:         Y
name:           ena
vermagic:       5.4.20-12.75.amzn2.x86_64 SMP mod_unload modversions 
sig_id:         PKCS#7
signer:         
sig_key:        
sig_hashalgo:   md4
signature:      30:82:02:9A:06:09:2A:86:48:86:F7:0D:01:07:02:A0:82:02:8B:30:
        82:02:87:02:01:01:31:0D:30:0B:06:09:60:86:48:01:65:03:04:02:
        01:30:0B:06:09:2A:86:48:86:F7:0D:01:07:01:31:82:02:64:30:82:
        02:60:02:01:01:30:3B:30:2E:31:2C:30:2A:06:03:55:04:03:0C:23:
        42:75:69:6C:64:20:74:69:6D:65:20:61:75:74:6F:67:65:6E:65:72:
        61:74:65:64:20:6B:65:72:6E:65:6C:20:6B:65:79:02:09:00:F8:0B:
        76:3F:91:73:04:A8:30:0B:06:09:60:86:48:01:65:03:04:02:01:30:
        0D:06:09:2A:86:48:86:F7:0D:01:01:01:05:00:04:82:02:00:61:C2:
        23:88:3A:67:56:25:02:02:D5:1A:27:64:4E:31:C3:2E:D5:33:A8:51:
        7E:AD:76:21:60:88:CF:F6:0C:72:EE:95:43:B5:D4:1A:07:79:5E:84:
        2F:F6:12:6C:66:44:A9:EC:66:73:FF:82:ED:B2:21:05:82:FC:60:3C:
        1C:BB:7D:58:2D:31:0A:C2:15:2E:66:87:D6:93:7A:EE:B2:09:D2:2F:
        63:66:0D:AA:74:C9:02:DA:0F:F5:68:8F:B7:DC:A7:0C:69:5D:31:24:
        69:30:96:06:33:2B:32:C8:A7:8A:5C:66:52:C8:04:42:D1:F9:AF:B2:
        5E:FB:E8:24:73:6C:5B:8D:59:69:F2:FE:BB:79:CC:47:15:41:3B:63:
        1D:42:9A:69:C9:B2:17:31:1E:94:0B:E7:1B:23:FB:75:B8:91:FA:6E:
        B5:45:45:CA:18:F8:CE:A4:46:C5:E3:19:72:90:69:11:1F:DF:F6:79:
        E4:67:4B:00:82:5B:61:7A:CE:DD:2A:69:09:38:01:27:92:C4:BF:83:
        21:9B:3F:B8:AB:44:6C:C0:7F:8B:F2:66:F0:B9:F3:57:7E:AE:B9:93:
        E4:AA:92:01:CE:AF:C0:89:16:DE:19:DB:39:FF:60:89:8E:5F:B0:94:
        B0:8C:DA:04:44:EC:74:05:E1:F5:52:09:72:EC:76:67:64:34:45:E6:
        34:BE:7A:8F:C9:3C:86:1A:EB:E8:26:C9:A0:13:8C:97:CE:3E:CC:A1:
        FE:FD:68:BA:74:3C:13:1E:EA:34:A0:2D:3B:24:AD:A2:0C:C9:91:45:
        63:28:64:FC:B5:DC:82:D2:45:B3:73:F5:59:BA:77:24:04:29:F9:AE:
        0D:4B:A1:6A:C7:D1:A2:6D:D0:29:E0:B6:0B:02:B2:DA:55:60:67:6C:
        6F:F1:98:28:6A:3E:58:96:D3:AB:0F:B1:68:99:F6:77:FE:6B:37:50:
        1F:14:E7:F0:F7:64:5A:0B:2C:18:C8:64:47:88:74:2B:44:5C:65:5D:
        27:B7:7E:65:D8:5C:C0:39:E9:53:3D:4E:90:6F:F1:D6:FE:8A:84:FA:
        55:A4:B9:EC:98:FA:77:BA:BB:E7:3E:29:AB:BF:B0:9B:DD:0B:CF:DE:
        84:C1:BA:F4:AD:B5:72:99:60:5E:F0:B5:C5:95:CE:FE:2C:1A:91:AF:
        96:D1:86:22:5C:7F:77:CE:90:B0:7A:5C:97:5E:FD:04:F5:4E:9A:D3:
        B0:50:A8:DC:11:B2:0F:24:CB:83:80:28:59:3B:A2:0E:0B:FF:60:36:
        26:45:AA:B4:04:9D:E0:D4:C0:F6:0B:9A:4A:E6:E3:66:E7:F5:E2:61:
        6E:4D:42:56:E8:14:32:F6:5A:19
parm:           debug:Debug level (0=none,...,16=all) (int)
parm:           rx_queue_size:Rx queue size. The size should be a power of 2. Max value is 8K
 (int)
parm:           force_large_llq_header:Increases maximum supported header size in LLQ mode to 224 bytes, while reducing the maximum TX queue size by half.
 (int)
parm:           num_io_queues:Sets number of RX/TX queues to allocate to device. The maximum value depends on the device and number of online CPUs.
 (int)

[sameehj]

Hi @jeff-2 ,

I haven't experienced your issue, I will try to reproduce ASAP. Meanwhile, can you try adding another interface to the instance (eth1) and try running the XDP pass program their. This will prevent you from getting disconnected while attempting to run it.

[ShayAgros]

Hi @jeff-2 , Sorry for not updating earlier. This patch should fix your issue. It would be part of the next driver release in a few weeks. The patch can be applied by running git am 0001-Bug-Fix-account-for-XDP-headroom-in-packet-s-page-of.patch.txt in the git repository of this driver.

Please update if it solves your issue

[jeff-2]

Thanks @ShayAgros - I can confirm that patch resolves the issue for me.

[akiyano]

Hi @jeff-2,

We've just released driver version 2.2.8g with the fix for this issue. Closing this issue.

Cheers, Arthur