search

go-spatial / tegola

Tegola is a Mapbox Vector Tile server written in Go
http://tegola.io/
MIT License
1.28k stars 194 forks source link

Caching + CORS questions #693

Open Seltzer opened 4 years ago

Seltzer commented 4 years ago

I'm looking into deploying Tegola via the Docker image and I have a few questions. Apologies if I've missed some docs where these are answered.

  1. Does Tegola itself evict entries from its file cache? I assume the cache grows indefinitely until there's no more disk space or tiles are explicitly purged.

  2. Would it be possible for two instances of Tegola (e.g. load balanced) to share a single file cache without issue?

  3. This is an extremely long shot but would you ever consider adding support for dynamic fetching of CORS origins (e.g. from a DB table or REST endpoint)? This has come up because we often provision new websites on the fly and we're trying to avoid having to manually add origins to the list and restart the container when launching a new site. We'll most likely solve this by shoving an in-container reverse proxy in front of Tegola and handling it at that level.

Cheers, Nathan

ARolek commented 4 years ago

@Seltzer Answers to your questions in order:

  1. Tegola does not evict files from the file cache (no TTL) so yes, it will keep creating tiles until they're explicitly purged or the disk runs out of space.
  2. I'm not quite sure what issues you would run into, but I do think there would be some edge cases. For example, the file cache does not have a "lock" concept right now, so technically a file could be in the process of being written while another instance tries to read it. There may be other situations that I'm not thinking of. @gdey & @ear7h might have some ideas of what to look out for.
  3. Hmm, this is not on the roadmap right now, but I'm interested in more details. Are you looking to have the CORS header set to the domain of the incoming request, or are you saying you have a dynamic whitelist of CORS domains that you're looking to have tegola include in the response headers?
Seltzer commented 4 years ago

Thanks for your responses.

Sounds like we should avoid trying to share the cache as we'd be relying on undefined behaviour in any case.

We have an ever-growing whitelist of CORS domains stored in a DB table which we're hoping to add to the response header. I'm not sure how many others are in a similar situation so it's hard to say whether it's justifiable to add it to the roadmap. I suspect the primary use-case it'd be catering for is when you have a multi-tenanted website with maps on it where each tenant has their own domain.

ARolek commented 4 years ago

@Seltzer I have not played with it personally, but the tegola s3 cache backend can be used against https://github.com/minio/minio. This might be a viable alternative to the filestore for your situation.

Regarding the whitelist, this sounds more application-specific so I think your idea of a proxy in front of tegola is probably the best idea. I'm happy to leave the conversation open in case anyone else wants to chime in, and I would love to hear what you end up with. I think others will encounter this problem as user adoption grows.