search

restatedev / restate

Restate is the platform for building resilient applications that tolerate all infrastructure faults w/o the need for a PhD.
https://docs.restate.dev
Other
1.61k stars 35 forks source link

Cache frequently accessed structures in `PartitionStore` #1873

Open tillrohrmann opened 2 months ago

tillrohrmann commented 2 months ago

To avoid the serialization cost when accessing data stored in the PartitionStore, we could add an LRU cache for frequently accessed data. Prime candidates could be the deduplication table and the InvocationStatus.

slinkydeveloper commented 2 months ago

Could this LRU cache be completely hidden behind the InvocationStatusTable impl? In particular once we get rid of these https://github.com/restatedev/restate/issues/276

tillrohrmann commented 2 months ago

I've ran some simple experiments and caching of the InvocationStatus didn't have a measurable impact on throughput.

slinkydeveloper commented 2 months ago

I've ran some simple experiments and caching of the InvocationStatus didn't have a measurable impact on throughput.

Did you by any chance tried with the neo invocation status to see if the new protobuf layout has any either positive or negative impact?

tillrohrmann commented 2 months ago

Did you by any chance tried with the neo invocation status to see if the new protobuf layout has any either positive or negative impact?

No, I only tried it with the old InvocationStatus. Given that reading the InvocationStatus took ~1.5% of time in the flamegraphs, I wouldn't expect a super noticeable change. This doesn't mean that the we shouldn't optimize these things, though.