when the latency is below certain threshold, a pushing dispatcher can be in a direct mode that bypass Queue and Worker and directly send the work to backend without any concurrency throttling.
When without any CPU pressure, kanaloa on average adds a 0~2 ms (see comparison below) overhead on a MacbookPro. But when the traffic is high CPU pressure could impose high penalty on latency (more than 100%) and throughput (more than 50%) , a direct mode is justified when CPU becomes bottleneck.
when the latency is below certain threshold, a pushing dispatcher can be in a direct mode that bypass Queue and Worker and directly send the work to backend without any concurrency throttling.
When without any CPU pressure, kanaloa on average adds a 0~2 ms (see comparison below) overhead on a MacbookPro. But when the traffic is high CPU pressure could impose high penalty on latency (more than 100%) and throughput (more than 50%) , a direct mode is justified when CPU becomes bottleneck.
Low loads Without kanaloa
Low load with kanaloa
High load without kanaloa (lots of CPU pressure)
High load with kanaloa (lots of CPU pressure)