Support for Arm64

[ztz1989]

Dear experts of openNetVM,

I have an Nvidia Bluefield DPU with Arm64 architecture. It runs Ubuntu 20.04.

I would like to deploy openNetVM on this device, do you provide support for Arm architecture?

Thanks & regards

[twood02]

We have not attempted to run OpenNetVM on arm64 recently. Previously (~3 years ago) we were able to compile and run the manager and sample NFs on arm64, but at the time the ARM drivers for mellanox cards did not support DPDK, so it could not be used to process real traffic. This may have changed by now. If you are able to test it, please let us know here if it works!

I don't think there should be anything specific to ONVM which would prevent it from running on ARM, so getting DPDK to work is the main limitation. That has probably been resolved by now.

[kkrama]

My student has tested Mellanox/nVidia Bluefield's support for DPDK. He is examining compatibility with OpenNetVM. I have cc'd him here. K. K. Ramakrishnan

On Thu, Nov 17, 2022 at 7:11 AM Tim Wood @.***> wrote:

We have not attempted to run OpenNetVM on arm64 recently. Previously (~3 years ago) we were able to compile and run the manager and sample NFs on arm64, but at the time the ARM drivers for mellanox cards did not support DPDK, so it could not be used to process real traffic. This may have changed by now. If you are able to test it, please let us know here if it works!

I don't think there should be anything specific to ONVM which would prevent it from running on ARM, so getting DPDK to work is the main limitation. That has probably been resolved by now.

— Reply to this email directly, view it on GitHub https://github.com/sdnfv/openNetVM/issues/332#issuecomment-1318779489, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC6TUJICWMBH7GDKLZGGSQLWIZDK7ANCNFSM6AAAAAAR7SW4GM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- K. K. Ramakrishnan

[ShixiongQi]

I have tested OpenNetVM (commit 20e0150) on NVIDIA Bluefield-2 and get it successfully run. There are several changes you need to make before getting the OpenNetVM work on Bluefield-2 (see the patch below).

Another change you need to make during the installation of OpenNetVM is to change the RTE_TARGET to arm64-bluefield-linuxapp-gcc. You can use the command below to enable this change.

echo export RTE_TARGET=arm64-bluefield-linuxapp-gcc >> ~/.bashrc

The remaining steps keep the same as the default installation guidance.

diff --git a/dpdk b/dpdk
--- a/dpdk
+++ b/dpdk
@@ -1 +1 @@
-Subproject commit e2a234488854fdeee267a2aa582aa082fce01d6e
+Subproject commit e2a234488854fdeee267a2aa582aa082fce01d6e-dirty
diff --git a/onvm/go.sh b/onvm/go.sh
index 4a84c7c..71e91e7 100755
--- a/onvm/go.sh
+++ b/onvm/go.sh
@@ -224,12 +224,12 @@ ports_bin="${ports_bin//0/}"
 # The number of ports is the length of the string of 1's. Using above example: 1111 -> 4
 count_ports="${#ports_bin}"

-ports_detected=$("$RTE_SDK"/usertools/dpdk-devbind.py --status-dev net | sed '/Network devices using kernel driver/q' | grep -c "drv")
-if [[ $ports_detected -lt $count_ports ]]
-then
-    echo "Error: Invalid port mask. Insufficient NICs bound."
-    exit 1
-fi
+#ports_detected=$("$RTE_SDK"/usertools/dpdk-devbind.py --status-dev net | sed '/Network devices using kernel driver/q' | grep -c "drv")
+#if [[ $ports_detected -lt $count_ports ]]
+#then
+#    echo "Error: Invalid port mask. Insufficient NICs bound."
+#    exit 1
+#fi

 # Trim 0x from NF mask
 nf_cores_trimmed=${nf_cores:2}
diff --git a/onvm/onvm_mgr/onvm_init.c b/onvm/onvm_mgr/onvm_init.c
index 25884dc..7b50dbc 100644
--- a/onvm/onvm_mgr/onvm_init.c
+++ b/onvm/onvm_mgr/onvm_init.c
@@ -122,7 +122,7 @@ static const struct rte_eth_conf port_conf = {
         },
     .rx_adv_conf = {
             .rss_conf = {
-                    .rss_key = rss_symmetric_key, .rss_hf = ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP | ETH_RSS_L2_PAYLOAD,
+                    .rss_key = rss_symmetric_key, .rss_hf = ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP | ETH_RSS_L2_PAYLOAD, .rss_key_len = 40,
                 },
         },
     .txmode = {.mq_mode = ETH_MQ_TX_NONE,
diff --git a/onvm/onvm_nflib/onvm_flow_table.h b/onvm/onvm_nflib/onvm_flow_table.h
index 6373aa0..9184653 100644
--- a/onvm/onvm_nflib/onvm_flow_table.h
+++ b/onvm/onvm_nflib/onvm_flow_table.h
@@ -89,7 +89,7 @@ union ipv4_5tuple_host {
                 uint16_t port_src;
                 uint16_t port_dst;
         };
-        __m128i xmm;
+        // __m128i xmm;
 };

 struct onvm_ft *

[ztz1989]

Dear all, I confirm that by following the changes of @ShixiongQi, openNetVM also works on my Bluefield DPU. Thanks for the support. I'll close this thread.