StationA / tilenol

Scalable, multi-backend geo vector tile server
MIT License
22 stars 6 forks source link

Add an option to auto-reload configuration file on change #41

Open jerluc opened 3 years ago

jerluc commented 3 years ago

There are some use cases in which it would make sense for tilenol to be able to auto-reload its configuration.

For example, when tilenol's configuration is mounted as a file from a ConfigMap in a Kubernetes environment, the ConfigMap may change when someone applies a new configuration via kubectl, but tilenol does not update since it only ever reads the configuration from disk on boot. Notably there are some temporary workarounds for this particular scenario, e.g. Reloader, but it might make more sense to simply add a new CLI flag, e.g. --reload, to support automatically watching for configuration file changes.

jerluc commented 3 years ago

Note that there are a couple of minor implementation wrinkles that would need to be addressed in order to support this new feature:

  1. Since configuration can affect which layers are accessible and the source data behind them, we will need to provide some safety mechanism around the implicit "pointer swap" that will need to occur when rehydrating layers from a freshly-reloaded configuration while concurrently servicing other in-flight requests; the simplest thing we could do is to use a multi-reader/single-writer mutex (the sync.RWMutex is probably most appropriate for this use case) to ensure that when the configuration is being applied to the running server, we at least retain memory consistency within a single request. Furthermore, since the configuration can also change the cache, this control mechanism/semaphore will need to lock readers before the caching middleware is invoked
  2. Since configuration changes may also imply that existing layers are reconfigured to return different data, we should also consider implementing the content-based hashing mechanism described in #25 as a way to forcibly cause cache misses for subsequent requests to updated layers