Trying to compile igb with double lan support

[deajan]

Hi, stumbled upon your project, trying to get a I211-AT to use QinQ.

When compiling your project, I get:

 make
make[1]: Entering directory '/usr/src/kernels/5.9.1-1.el8.elrepo.x86_64'
  CC [M]  /opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.o
/opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.c: In function 'igb_xmit_more':
/opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.c:5959:12: error: 'const struct sk_buff' has no member named 'xmit_more'
  return skb->xmit_more;
            ^~
/opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.c: In function 'igb_clean_tx_irq':
/opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.c:7721:3: error: implicit declaration of function 'read_barrier_depends'; did you mean 'rcu_barrier_tasks'? [-Werror=implicit-function-declaration]
   read_barrier_depends();
   ^~~~~~~~~~~~~~~~~~~~
   rcu_barrier_tasks
/opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.c: In function 'igb_xmit_more':
/opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.c:5963:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:283: /opt/install/igb-igb-5.3.6-double-vlan/src/igb_main.o] Error 1
make[1]: *** [Makefile:1784: /opt/install/igb-igb-5.3.6-double-vlan/src] Error 2
make[1]: Leaving directory '/usr/src/kernels/5.9.1-1.el8.elrepo.x86_64'
make: *** [Makefile:86: default] Error 2

This happened on CentOS 8 x64, with both redhat vanilla kernel kernel-modules-4.18.0-193.19.1.el8_2.x86_64 and newer mainline kernel-ml-5.9.1-1.el8.elrepo.x86_64, with gcc 8.3.1.

Anything I can provide to help diag ?

Best regards.

[serhepopovych]

I think I should rebase against new version of igb 5.4.6. This will take some time.

[serhepopovych]

Update to igb 5.4.6 didn't help. Need to address this. Looks like there is no ->xmit_more field in struct sk_buff in newer kernels. Working on this....

[serhepopovych]

For history: struct sk_buff->xmit_more bitfield replaced with ->recursion and ->more fields in struct softnet_data with these commits in upstream kernel:

dc93e85f9b10215c61433179f0f03e1d1e4b838 Merge branch 'xmit_more-softnet_data' 4f296edeb9d4cf76b876869461a7ae627c307110 drivers: net: aurora: use netdev_xmit_more helper f79c957a0b537d6871a8aa195d5b5bcc7e480957 drivers: net: sfc: use netdev_xmit_more helper 3c31ff22b25f15c6a642bb775884a599379a3cb5 drivers: mellanox: use netdev_xmit_more() helper 6b16f9ee89b8d5709f24bc3ac89ae8b5452c0d7c net: move skb->xmit_more hint to softnet data 97cdcf37b57e3f204be3000b9eab9686f38b4356 net: place xmit recursion in softnet data

Inline helper netdev_xmit_more() was introduced with this series. Intel uses preprocessor #if/#endif to define fallback macro for earlier kernels. Our code should now use this helper/preprocessor define instead of custom coded src/igb_main.c::igb_xmit_more().

These commits merged to 5.2-rc1 according to git history.

Need to address this.

[serhepopovych]

Please try igb-5.4.6/double-vlan or igb-5.4.6/netctl-tune (if you are using DKMS packaging).

Tested with stock CentOS 8.2 kernel 4.18.0-193.19.1.el8_2.x86_64 and DKMS. At least build works. Should also build with elrepo mainline kernel.

[deajan]

Hello,

Thanks for your answer. I could indeed compile igb-5.4.6/double-vlan against stock kernel 4.18.0-193.19.1.el8_2.x86_64. For now, I could activate the function via ethtool

ethtool --show-priv-flags enp7s0
Private flags for enp7s0:
vlan-stag-rx               : on
vlan-stag-filter           : on
vlan-stag-ethertype-802.1ad: on

I now have to "play" with the implementation and see whether I get what I need (eg using a couple of I211-AT cards connected to a bridge in order to emulate a 802.1ad edge switch).

Again, thank you for your time.

For info, compiling against latest elrepo kernel 5.9.1-1.el8.elrepo.x86_64 fails, for both double-vlan and netctl-tune versions:

make[1]: Entering directory '/usr/src/kernels/5.9.1-1.el8.elrepo.x86_64'
  CC [M]  /opt/install/igb-igb-5.4.6-double-vlan/src/igb_main.o
/opt/install/igb-igb-5.4.6-double-vlan/src/igb_main.c: In function 'igb_clean_tx_irq':
/opt/install/igb-igb-5.4.6-double-vlan/src/igb_main.c:7718:3: error: implicit declaration of function 'read_barrier_depends'; did you mean 'rcu_barrier_tasks'? [-Werror=implicit-function-declaration]
   read_barrier_depends();
   ^~~~~~~~~~~~~~~~~~~~
   rcu_barrier_tasks
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:283: /opt/install/igb-igb-5.4.6-double-vlan/src/igb_main.o] Error 1
make[1]: *** [Makefile:1784: /opt/install/igb-igb-5.4.6-double-vlan/src] Error 2
make[1]: Leaving directory '/usr/src/kernels/5.9.1-1.el8.elrepo.x86_64'
make: *** [Makefile:86: default] Error 2

[serhepopovych]

Yes, private flags seems to be correct. However I haven't tried if feature works on I211-AT as do not have such hardware. Suppose RSS will work correctly and load will be distributed across multiple queues as expected.

I fixed build for recent kernels and tested build with 5.9.2-1.el8.elrepo.x86_64. Issue also can be reproduced with original igb-5.4.6 and caused by missed commit c4cb99185b4c ("igb: Use smp_rmb rather than read_barrier_depends") from linux-stable kernel in igb-5.4.6 OOT driver. Fix backported with https://github.com/serhepopovych/igb/commit/3f27750561681377b301653674688f36135b4e4b in igb-5.4.6/fixes on which igb-5.4.6/double-vlan and igb-5.4.6/netctl-tune is based.

Thank you for bug report. Feel free to open new issues as you found them.

[serhepopovych]

Build issue should be resolved.