erwan
commented
9 years ago Hi Bart,
Yes indeed, the way we're caching now is not optimal because we expire all document each time a single change occur. This is something we're aware of.
We're working towards a better cache, where documents expire only when they are actually modified, but we have some work to do server-side. This is precisely because we still don't have this distinction (modified/unchanged documents in a release) that the webhook don't give more information.
As soon as the API is updated we'll update the kits.
tremby
Every time the ref changes, all the cached queries become invalid. This is far from ideal, especially for large sites with large numbers of unique queries. (For one of our sites, Edmonton Tourism, we get around this by prefetching every single document, and caching "fake" result sets for queries for each separate document, each group of documents by mask, each document by category if they're categorized, and so on. This is really wasteful but the site is far, far faster because of it.)
It would be very nice if documents could be cached separately and reused without refetching if they have not been changed (even if the ref has changed). For example if document A has been updated but B, C and D haven't, and they were all previously fetched and cached, it would be nice if only document A had to be fetched again rather than the cached copies of B, C and D being deemed invalid since they were fetched with an older ref.
Is this in the realm of possibility?
I can imagine how to separately cache documents -- that'd be easy enough.
If the API then allowed for only the IDs of matching documents to be returned for a given query, along with the ref at which they were last updated, these two could make up the cache keys for each document. The SDK could then look in the cache for those it still has -- these are still fresh and should not be modified. It then makes a second request to Prismic for all the other results, receives them and populates the cache.
Does such a "last updated ref" exist, per document, and can it be retrieved?
This could be overridden of course to simply get all full results, for instance if the developer knows the cache is empty or close to empty at this point.
It'd then be possible to precache all documents, without literally grabbing all data for all documents each time we get a webhook. (Webhooks don't include the specific IDs of documents which have changed -- it'd be nice if they had this too.)
I have a lot more thoughts on this, but this should be enough to get a discussion started.