tarantool / vshard

The new generation of sharding based on virtual buckets
Other
100 stars 31 forks source link

Implement router_call as a C-stored procedure using C++ connector #239

Open Buristan opened 4 years ago

Buristan commented 4 years ago

According to flamegraphs of vshard's routers by @alyapunov approximately 30% time of tx thread is wasted by decoding/encoding of msgpack to Lua and back.

We can use buffer option at net.box to send original msgpack including tuple itself without extra decode/encode to Lua, after reading necessary information in C(bucket_id), to improve router's perfomance.

Korablev77 commented 3 years ago

In scope of this issue, it was decided to implement router_call* function(s) as C stored procedures. Instead of using netbox connections, we are going to try new C++ connector's facilities (establish auxiliary connection, send requests without unpacking original tuple, receive responses and form resulting tuples). A bit later I'll attach brief RFC. After PoC is done and if we see significant performance benefits, it is worth considering implementing vshard router as a standalone application. It becomes possible owing to mentioned C++ connector (the only missing part is server-side API). I've already come up with code snippets and prototypes in C++ , but they seem to be raw. So I want to polish them a bit and public in separate repository.