search

pocketnetteam / pocketnet.core

Decentralized social network based on the blockchain
https://pocketnet.app
Apache License 2.0
114 stars 28 forks source link

Caching at the level of the RPC controller + repository database. #75

Open andyoknen opened 2 years ago

andyoknen commented 2 years ago

The main idea is to store in the cache not only the result of API methods with the key from the request input data - in practice it turned out that such a cache covers barely 30% of requests.

Therefore, I suggest implementing the cache deeper in API controllers and even in repositories. Below I will show an example:

https://github.com/pocketnetteam/pocketnet.core/blob/4cdb1631b7042b5f7c818a7a40789186a63c6e2c/src/pocketdb/repositories/web/WebRpcRepository.cpp#L2198

The GetHistoricalFeed function gets a set of post ids from the database by the right filter and with the right sorting.

After executing the main request, which may contain a rather specific set of filters, a single method for constructing JSON content for issuing to the client is launched - auto contents = GetContentsData(ids, address);

https://github.com/pocketnetteam/pocketnet.core/blob/4cdb1631b7042b5f7c818a7a40789186a63c6e2c/src/pocketdb/repositories/web/WebRpcRepository.cpp#L2282

And that's where the cache comes in handy - we can find all the identifiers (int64_t) in it and take the data we need from there, and take everything missing from the database.

The GetContentsData method is also not quite simple, it contains additional logic inside, for example, a selection of the last comment to the post and a short user profile - this data will also fit very well in the cache.

In general, repositories and controllers are not yet fully adapted to such a mechanism of work, but a start has been made and we can painlessly change the logic from version to version - without risking breaking anything.

The main requirement of the caching class, I think, is the ability to delete data not with each block, but during indexing of the block, based on the data set in the block. Any content editing or rating changes should lead to clearing the cache for the entities involved. We can implement the purification logic in several places, but due to the fact that each entity in the database has a unique digital identifier - we don't even need to worry about types - key = ID

the-real-vortex-v commented 2 years ago

Is this why the side panel hasn't been updating for the last month or more? I have to refresh the page in the webbrowser to get the last comments to refresh. A few months ago (At least 3) it was upto updating the comments.

andyoknen commented 2 years ago

At the moment, the cache is not implemented in the node. But there is a cache on proxy servers that stand between the frontend and the node.

andyoknen commented 2 years ago

Simple cache implementation for the RPC API - #202