ON-12878: use the new cplane user API

[jfeather-amd]

Starting with Xilinx-CNS/onload@b660ac5b259f33cddcb1c6bd6f81ef7910dc10a7, @rhughes-xilinx added a public cplane API to onload. This PR modifies tcpdirect to use this new public cplane API, and is ported from before the open-sourcing of tcpdirect. For internal reviewers, the following PRs may be useful reading as an introduction to this PR:

• Xilinx-CNS/tcpdirect_history#226, the original version of this PR in the archived closed-source tcpdirect.
• Xilinx-CNS/onload_internal#1332, the addition of the public cplane API to onload.

Many of the changes from the original PR are rebase conflicts, and some small bugs (mostly in the unit tests).

Testing Done

• [x] Unit tests are happy (locally and on GitHub actions)
• [x] Send to multicast address
  • [x] Normal (ZF_ATTR="interface=enp4s0f0" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.129.192:8080 239.1.2.3:8080)
  • [x] VLAN (ZF_ATTR="interface=enp4s0f0.1" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.111.1:8080 239.1.2.3:8080)
• [x] Send to unicast address
  • [x] Normal (ZF_ATTR="interface=enp4s0f0" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.129.192:8080 192.168.129.191:8080)
  • [x] VLAN (ZF_ATTR="interface=enp4s0f0.1" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.111.1:8080 192.168.111.2:8080)
• [x] Send over bond
  • [x] LACP
    • [x] Unicast to other bond (ZF_ATTR="interface=bond0" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.192.1:8080 192.168.192.2:8080)
    • [x] Multicast (ZF_ATTR="interface=bond0" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.192.1:8080 239.1.2.3:8080)
    • [x] Handle bonding changes
    • [x] VLAN unicast (ZF_ATTR="interface=bond0.1" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.111.1:8080 192.168.111.2:8080)
    • [x] VLAN multicast (ZF_ATTR="interface=bond0.1" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.111.1:8080 239.1.2.3:8080)
    • [x] Verify hashing works (layer3+4 by varying ports)
  • [x] active-backup
    • [x] Unicast to other bond (ZF_ATTR="interface=bond0" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.192.1:8080 192.168.192.2:8080)
    • [x] Multicast (ZF_ATTR="interface=bond0" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.192.1:8080 239.1.2.3:8080)
    • [x] Handle bonding changes (active slave changing)
    • [x] VLAN unicast (ZF_ATTR="interface=bond0.1" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.111.1:8080 192.168.111.2:8080)
    • [x] VLAN multicast (ZF_ATTR="interface=bond0.1" ./build/gnu_x86_64/bin/zf_apps/static/zfsend -i 1 192.168.111.1:8080 239.1.2.3:8080)

Perhaps worth noting, as I'm not sure if this behaviour was present before, but specifying a physical interface for transmit can be used to bypass logical interfaces like bonds and VLANs.

TODO

Some further modification to onload was required for me to get the test apps working (zfsend in this case), perhaps because of my specific workflow. I had to modify onload_install to also install the cplane libraries, otherwise tests would fail with ENOENT. The diff for this change is below, but I also intend to raise this as a PR in onload, unless someone points out an alternative making the below unecessary.

diff --git a/scripts/onload_install b/scripts/onload_install
index fa04b05cb2..e104b001c8 100755
--- a/scripts/onload_install
+++ b/scripts/onload_install
@@ -229,6 +229,8 @@ install_userland() {
               "$i_lib64/libonload.so"
     install_dlib "$u64"/lib/onload_ext/libonload_ext.so "$i_lib64"
     install_slib "$u64"/lib/onload_ext/libonload_ext.a "$i_lib64"
+    install_dlib "$u64"/lib/cplane/libefcp.so "$i_lib64"
+    install_slib "$u64"/lib/cplane/libcplane0.a "$i_lib64"
     if [ -n "$ppc_at" ]; then
       install_plib "${u64}_at/lib/transport/unix/libcitransport0.so" \
                    "$i_prefix/$ppc_at/lib64/libonload.so"

[rhughes-xilinx]

Can you look again at why we need libcplane0.a from onload? Surely the whole point of this change is to get rid of that

[jfeather-amd]

Can you look again at why we need libcplane0.a from onload? Surely the whole point of this change is to get rid of that

Is this regardng the TODO section onload diff? If so, I hadn't put much thought into the change, and just installed any libraries that contained the word "cplane". I would be very unsurprised if removing libcplane0.a from the install script had no effect on the final behaviour. The test apps only complains about the lack of libefcp.so.

[jfeather-amd]

After some deeper testing, including some performance testing (see ON-1440 internally), it looks like this patch is ready to go so I'll merge this now.