tomerdbz
commented
2 weeks ago Some sample results: Samples_for_XLIO_CQ_AIM_INTERRUPTS_RATE_PER_SEC.csv
Closed tomerdbz closed 2 weeks ago
tomerdbz
commented
2 weeks ago Some sample results: Samples_for_XLIO_CQ_AIM_INTERRUPTS_RATE_PER_SEC.csv
tomerdbz
commented
2 weeks ago bot:retest
Better default
XLIO_CQ_AIM_INTERRUPTS_RATE_PER_SECfor the general use-case.Description
With Nginx scenarios when downloading small files (0B payloads) alongside
XLIO_CQ_MODERATION_ENABLE=1- a throughput degradation was observed. This, in contrast to running the same scenarios withXLIO_CQ_MODERATION_ENABLE=0.The degradation was 18% for 1200 connections.
Playing with the default value of
XLIO_CQ_AIM_INTERRUPTS_RATE_PER_SECfor 0B, we managed to cancel the degradation. We also checked the new default does not hurt the 100KB use case.What
Change the default of
XLIO_CQ_AIM_INTERRUPTS_RATE_PER_SECfor the general use-case.Why ?
This PR solves bug 3914075, canceling the degradation observed for small payload download. The above has a neglectable, 0.06% degradation in the 100KB case.
How ?
I printed the arguments to
ring_simple::modify_cq_moderation, witnessing them momentarily spiking. upon ignoring the spikes by putting constant values, the performance improved.We've witnessed the arguments are controlled by
XLIO_CQ_AIM_INTERRUPTS_RATE_PER_SEC- which is 1000 by default. I've conducted an extensive matrix of benchmarks - checking:The results were mixed, but it was clear every set of (connections, file_length) had a range of good interrupt rate values - and a set of really bad values.
The original thinking was finding it dynamically, to adjust the interrupt rate based on the packet rate. This gave best results across all use cases https://github.com/Mellanox/libxlio/pull/253.
But! the matrix also revealed enlarging the default, w/o further work - would be very good for the general case. We chose the simpler solution now.
Change type
What kind of change does this PR introduce?