search

NetSys / bess

BESS: Berkeley Extensible Software Switch
Other
313 stars 156 forks source link

samples/qtest.bess: Priority anomaly #865

Closed levaitamas closed 5 years ago

levaitamas commented 6 years ago

Hi!

There is an interesting phenomenon in qtest.bess.

The fast task gets starved by the queue_rr task, even though fast has a higher priority than queue_rr.

localhost:10514 $ run samples/qtest
Done.
localhost:10514 $ monitor tc
Monitoring traffic classes: !leaf_source1:0, queue_rr, fast, w2root, slow, w1root, !leaf_source0:0, !leaf_queue:0

15:25:44.772068          CPU MHz   scheduled        Mpps        Mbps  pkts/sched    cycles/p
--------------------------------------------------------------------------------------------
W1 !leaf_source1:0         0.000           0       0.000       0.000       0.000       0.000
W1 queue_rr             2607.100    24699713       0.000       0.086       0.00020454352.773
W1 fast                    0.000           0       0.000       0.000       0.000       0.000
W1 w2root               2610.505    24702985       0.000       0.086       0.00020478397.641
W0 slow                    0.099           3       0.000       0.086      32.000     776.375
W0 w1root                  0.099           3       0.000       0.086      32.000     776.375
W0 !leaf_source0:0         0.099           3       0.000       0.086      32.000     776.375
W1 !leaf_queue:0        2611.494    24712338       0.000       0.086       0.00020478397.641
--------------------------------------------------------------------------------------------
localhost:10514 $

Switching priorities mitigates the starvation.

localhost:10514 $ run samples/qtest-switch-prio
Done.
localhost:10514 $ monitor tc
Monitoring traffic classes: !leaf_source1:0, fast, queue_rr, w2root, slow, w1root, !leaf_source0:0, !leaf_queue:0

15:28:06.560009          CPU MHz   scheduled        Mpps        Mbps  pkts/sched    cycles/p
--------------------------------------------------------------------------------------------
W1 !leaf_source1:0         2.989        3122       0.100      67.146      32.000      29.918
W1 fast                    2.989        3122       0.100      67.143      32.000      29.917
W1 queue_rr             2597.039    23980186       0.000       0.064       0.00027145546.677
W1 w2root               2600.035    23983316       0.100      67.207       0.004   25997.708
W0 slow                    0.062           2       0.000       0.064      32.000     651.760
W0 w1root                  0.062           2       0.000       0.064      32.000     651.760
W0 !leaf_source0:0         0.062           2       0.000       0.064      32.000     651.760
W1 !leaf_queue:0        2597.049    23980602       0.000       0.064       0.00027148741.698
--------------------------------------------------------------------------------------------
localhost:10514 $

The situation seems strange since the BESS Scheduler Wiki page mentions that 'the node always schedules the child with the highest priority that's not blocked'.

levaitamas commented 5 years ago

Okay, The BESS Scheduler Wiki page also mentions that 'The priority parameter control the priority that this node has among the children of a priority parent (a lower number means higher priority).' Oops.. my bad.

Back to qtest.bess: there are queue_rr (prio:0) and fast (prio:1) tc-s in w2_root (policy='priority'). The queue_rr tc will be scheduled first due to its priority and never gets blocked since it is a lonely member of a round_robin tc and has no rate_limit or other limitation so it can eat up all the CPU and starve the fast tc.

Setting the priority of fast to 0 indeed solves the issue since fast will be blocked intermittently due to its rate limitation thus queue_rr will not get starved.

I send a fix PR soon.