Request logging

[davenquinn]

One of the oldest issues in this repository pertains to the need to show metrics about tile usage #4 . We've actually been capturing statistics about tile usage since the inception of the tileserver v1. Basically,

• Each request is logged into a PostgreSQL tileserver_stats database from within the tileserver code.
• A batch script in the tileserver stats repository creates statistics and truncates the log at some interval (currently manually run).

It would be fairly straightforward to capture similar statistics from the v2 tileserver, with a few caveats:

• We will need to handle a slightly wider array of distinguishing information such as which route a tile was served from
• It would be handy to log which maps were served, but this can be inferred later if necessary
• We have moved to HTTP caching (varnish) over in-memory storage (Redis) for simplicity. However, this means that cached requests cannot be logged using the same approach as db-served requests. This will result in our logs ignoring repeated requests.

Perhaps there's an easy way to adapt varnish logging into this system? Alternatively, we can consider whether to move back to Redis to allow more unified cache handling, at the cost of some additional application complexity.

[davenquinn]

This is a key part of ensuring that we have a good analytics setup in Macrostrat v2, UW-Macrostrat/macrostrat#67.

[davenquinn]

Some information on Varnish logging: https://varnish-cache.org/tips/varnishlog/index.html

[davenquinn]

Per @CannonLock it sounds like we will be handling tileserver logs at the cluster level via Traefik, and parsing inserting into our tileserver logs database.