When Congestion occurs with shared core, number of pkt processing is abnormal

[JackKuo-tw]

Bug Report

Current Behavior

When there are 2 NFs share with a single CPU core, if the total CPU usage goes up to 100%, the packet drops reasonably.

But in my test, the tx_pps & rx_pps drop drop sharply,

Expected behavior/code

In this scenario, CPU usage is only about 70%, controlled by pktgen's packet per second.

Steps to reproduce

• 2 NFs
  • simple_forward forwards pkt to firewall
  • sudo ./build/simple_forward -l 5 -n 3 --proc-type=secondary -- -r 1 -s -m 5 -- -d 2
  • firewall drops all the pkts
  • sudo ./build/app/firewall -l 5 -n 3 --proc-type=secondary -- -r 2 -s -m 5 -- -d 1 -f rules.json
• Pktgen
  • generate UDP pkts using the default configuration as ONVM provides

Environment

• OS: Ubuntu 20.04 LTS
• onvm version: v20.10
• dpdk version: 20.05.0

Possible Solution

I cannot find the core problem...

Additional context/Screenshots

[dennisafa]

Thanks for the bug report. I think that one way to solve this is to increase the RX queue size for the NF's. Since the CPU utilization is high, the NF cannot process the current batch of packets at a high enough rate before the next batch comes in from pktgen. This results in drops since the RX queue is full.

[dennisafa]

Were you able to solve this issue, or do you require further assistance?

[JackKuo-tw]

@dennisafa I tried to change the value of NF_MSG_QUEUE_SIZE from 128 to 65536 in onvm_mgr/onvm_init.h, but it seemed nothing happend.

Maybe the flow is still too large, this change can only afford the burst (not test yet)

[JackKuo-tw]

Is there any fundamental way to solve this issue?

[dennisafa]

@dennisafa I tried to change the value of NF_MSG_QUEUE_SIZE from 128 to 65536 in onvm_mgr/onvm_init.h, but it seemed nothing happend.

Maybe the flow is still too large, this change can only afford the burst (not test yet)

Thats a different queue size constant - it's for inter-nf messages, not packets. There should be a different constant that is the RX queue size. Here it is: https://github.com/sdnfv/openNetVM/blob/master/onvm/onvm_nflib/onvm_common.h#L71 @JackKuo-tw

[JackKuo-tw]

@dennisafa Thanks. It doesn't work for me. BTW, ringsize is limited to unsigned (4 bytes), but millions of packets per second, does it matter?