search

WordPress / openverse-api

The Openverse API allows programmatic access to search for CC-licensed and public domain digital media.
https://api.openverse.engineering/v1
MIT License
77 stars 51 forks source link

Set up a `tallies` cache alias for tracking longer-term values #1072

Closed AetherUnbound closed 1 year ago

AetherUnbound commented 1 year ago

Problem

We are presently using the default django.core.cache alias for tracking view caches & authentication information in addition to general tallies and aggregates we wish to track. This means that we have both cache and (ostensibly) longer-term data living in the same Redis database, and (unless explicitly adjusted) existing under the same TTLs.

Description

We would like to set up a separate cache alias, tallies, which could be used to store longer-living values. The two examples that come to mind would be the dead link tallies that are accumulated in validate_images, and the thumbnail domain timeouts that were added in #982.

This alias could have a longer (or nonexistent) TTL, with the option to define individual timeouts when writing to the cache.

We should also check through other uses of the cache and swap any uses that are not view caches & authentication info to use tallies.

sarayourfriend commented 1 year ago

This alias could have a longer (or nonexistent) TTL, with the option to define individual timeouts when writing to the cache.

I think it should specifically have no TTL for this usecase and if entries need to be timestamped (or a record kept of "last updated") then secondary keys can be used to accomplish that.

The other use of cache that exists in the validate images function is the dead link mask. This needs to retain its existing behaviour and uses a hand-configured TTL. Just want to note that for whomever works on this issue: be sure to keep an eye out for not accidentally changing the behaviour of the dead link masks :slightly_smiling_face: