anycable / graphql-anycable

A drop-in replacement for GraphQL ActionCable subscriptions. Works with AnyCable.
MIT License
112 stars 18 forks source link

Stale subscriptions data cleanup for operational stability #1

Open Envek opened 6 years ago

Envek commented 6 years ago

As this gem actually stores runtime data in Anycable's Redis instance it is comparatively easy to use all available space by stale data about subscriptions (if for some reason it weren't deleted on client unsubscription).

One possible variant is:

  1. Set expiration with some configurable timeout (with default with 1 day) for subscriptions (SUBSCRIPTION_PREFIX + subscription.id keys). Let it die. But events keys can't be expired as it will lead to broken subscriptions.
  2. On application (anycable RPC server) start subscribe to all expire events in Redis via Keyspace notifications (but in that case subscription_id → event_id mapping should be stored separately from subscription itself as it is impossible to retrieve contents of an expired key)
  3. On key expired event fired subscription_id should be removed from all events. Event keys from which the last element was removed will be deleted automatically by Redis.
  4. Additionally, it is possible to run some rake task every day or week and check every SUBSCRIPTION_PREFIX keys and check their staleness via OBJECT IDLETIME, remove their ids from every event and remove them altogether.

The problem of this approach is in Redis configuration that should be altered to let it work: Keyspace notifications are disabled by default and OBJECT command may be disabled by security concerns (as it is considered to be a service command for debug purposes). And there may be no privileges to change these settings.

Any other ideas?