RediSearch / RSCoordinator

RediSearch coordinator
Other
23 stars 10 forks source link

[PERF] 40% of RSCoordinator background cpu time is spent on sequential (n shards) network buffer write #305

Open filipecosta90 opened 2 years ago

filipecosta90 commented 2 years ago

Given we write to each N shards one after the other the _FT.SEARCH, this is ultimately costing us 40% of total CPU cycles and impacts the thread capabilities of parsing the requests replies and other required logic. Meaning the writes are slow and impacts everything after it.

image

Sharing the main and background thread flame charts with the #303 already solved so we focus on the new real hotspots. The first thing to notice is that the coordinator thread is the first bottleneck ( reaching 100% CPU usage faster than the main thread ).