go-spatial / tegola

Tegola is a Mapbox Vector Tile server written in Go
http://tegola.io/
MIT License
1.3k stars 198 forks source link

cache: allow to set a local proxy endpoint for s3 #992

Closed elukey closed 5 months ago

elukey commented 6 months ago

Hi folks!

At Wikimedia we use Tegola on Kubernetes and recently we noticed a performance regression when the internal S3 endpoint changed its TLS certificate (more info https://phabricator.wikimedia.org/T344324). We currently use version 0.19.x, we haven't tested the more recent ones, but we'd like to use a local sidecar/proxy to improve HTTP connection pooling and similar settings.

We are currently connecting from the pod directly to the internal S3 endpoint, and we may explore other alternatives if you don't like the one proposed in this patch. The only one that comes to mind is adding explicit settings to tune HTTP connection pooling and similar in tegola's AWS sdk config for S3.

Technical details: If Tegola is configured to use a s3 cache endpoint like http://localhost:1234, the AWS request signing step is not done correctly and the real S3 endpoint will return a HTTP 403 stating that the request signing header value doesn't match what it expects. This is a typical use case in k8s, where the HTTP connection go through a local sidecar/proxy. In our case, we have an internal S3 endpoint (implemented via Openstack Swift) and we noticed an increase in CPU usage in the pod running tegola when the TLS cert of the S3 endpoint changed (probably due to high requirements for the Cipher suite). We would like to implement in the local proxy connection pooling and other similar performance improvements, without necessarily changing any of the Tegola's code or settings. We'd also get HTTP metrics for free from the sidecar, without the need of instrumenting Tegola.

Let me know your thoughts! Thanks in advance :)

coveralls commented 6 months ago

Pull Request Test Coverage Report for Build 86cf1919d-PR-992

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
cache/s3/s3.go 10 23 43.48%
<!-- Total: 10 23 43.48% -->
Totals Coverage Status
Change from base Build 5563b2dd0: -0.009%
Covered Lines: 6551
Relevant Lines: 14024

💛 - Coveralls
elukey commented 6 months ago

Hi @ARolek! When you have a moment: what do you think about the PR? It is running in Wikimedia production at the moment, looks working as expected :)

ARolek commented 6 months ago

👋 @elukey I will have some time tomorrow to give this a review. Thanks for your submission!