apenwarr
commented
4 years ago I think the new ActivityLog support actually gives a really nice option for caching. Basically, notion provides the latest version of every block that a) has changed or b) is directly referred to by one of the changed blocks. By incrementally updating the local copy of the ActivityLog and caching the latest received version of each block, we should be able to keep the local database in sync without having to constantly fetch a list of page versions. This seems to be working pretty well so far in https://github.com/apenwarr/notionchanges, although at the moment I mostly only care about block titles, types, and hierarchy positions. ᐧ
On Sun, Jun 28, 2020 at 2:05 AM Krzysztof Kowalczyk < notifications@github.com> wrote:
FYI: regarding caching, I used to cache responses as JSON but I found it fragile.
In my projects I'm using caching_downloader (sub-package in the repo) which does caching on the level of HTTP requests.
The basic idea is: I wrote caching_http_client which can cache GET / POST requests in an abstract cache (there's implementation for cache using directory).
caching_downloader adds some convenience wrapper around it. It can intelligently re-download a page only if it changed (notion has a revision number for each block and an API to ask for latest version, so I can cheaply ask for changed pages and only re-download those).
It's not documented but you can see https://github.com/kjk/blog for example of how I do it ( https://github.com/kjk/blog/tree/master/notion_cache is directory with cached http requests).
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/kjk/notionapi/pull/26#issuecomment-650702449, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAFA4GBWS5UCY5ISJY4OM3RY3MRBANCNFSM4OKJL75Q .
-- Avery Pennarun // CEO @ Tailscale
FYI: regarding caching, I used to cache responses as JSON but I found it fragile.
In my projects I'm using
caching_downloader(sub-package in the repo) which does caching on the level of HTTP requests.The basic idea is: I wrote
caching_http_clientwhich can cache GET / POST requests in an abstract cache (there's implementation for cache using directory).caching_downloaderis a convenience wrapper around it. It can intelligently re-download a page only if it changed (notion has a revision number for each block and an API to ask for latest version, so I can cheaply ask for changed pages and only re-download those).It's not documented but you can see https://github.com/kjk/blog for example of how I do it (https://github.com/kjk/blog/tree/master/notion_cache is directory with cached http requests).