search

restatedev / restate

Restate is the platform for building resilient applications that tolerate all infrastructure faults w/o the need for a PhD.
https://docs.restate.dev
Other
1.42k stars 34 forks source link

Cache frequently accessed structures in `PartitionStore` #1873

Open tillrohrmann opened 3 weeks ago

tillrohrmann commented 3 weeks ago

To avoid the serialization cost when accessing data stored in the PartitionStore, we could add an LRU cache for frequently accessed data. Prime candidates could be the deduplication table and the InvocationStatus.

slinkydeveloper commented 3 weeks ago

Could this LRU cache be completely hidden behind the InvocationStatusTable impl? In particular once we get rid of these https://github.com/restatedev/restate/issues/276

tillrohrmann commented 3 weeks ago

I've ran some simple experiments and caching of the InvocationStatus didn't have a measurable impact on throughput.

slinkydeveloper commented 3 weeks ago

I've ran some simple experiments and caching of the InvocationStatus didn't have a measurable impact on throughput.

Did you by any chance tried with the neo invocation status to see if the new protobuf layout has any either positive or negative impact?

tillrohrmann commented 3 weeks ago

Did you by any chance tried with the neo invocation status to see if the new protobuf layout has any either positive or negative impact?

No, I only tried it with the old InvocationStatus. Given that reading the InvocationStatus took ~1.5% of time in the flamegraphs, I wouldn't expect a super noticeable change. This doesn't mean that the we shouldn't optimize these things, though.