Add a mechanism to retrieve a signed URL from filestore and redirect to it, instead of proxying the file directly.
Falls back to proxying if the url can't be retrieved.
Related Issues / PRs
Fixes overleaf/issues#2937
Review
I've only added support for redirects to the GCS persistor, as this is where the bulk of the traffic goes. If we're using the S3 persistor now it's because we're falling back to an unusual bucket.
It's running in staging with redirect URLs only enabled for filestore-readonly. I think this is sensible for now as that's where the bulk of the load comes from. Compiles still work and I'm seeing hits on the file_redirect metric.
I couldn't get acceptance tests working against this, or signed URLs working in dev. Tested in staging.
Potential Impact
Requires a settings change to proxy the files - assuming we're just making this change for readonly, this affects compiles.
Manual Testing Performed
[x] Tested compiles in staging with this setting turned on
Deployment
Requires setting ALLOW_REDIRECTS: 'true' for filestore-readonly
Metrics and Monitoring
There are two new metrics: file_redirect and file_redirect_error so that we can watch for problems and make sure that it's doing what we want.
Description
Add a mechanism to retrieve a signed URL from filestore and redirect to it, instead of proxying the file directly.
Falls back to proxying if the url can't be retrieved.
Related Issues / PRs
Fixes overleaf/issues#2937
Review
I've only added support for redirects to the GCS persistor, as this is where the bulk of the traffic goes. If we're using the S3 persistor now it's because we're falling back to an unusual bucket.
It's running in staging with redirect URLs only enabled for
filestore-readonly. I think this is sensible for now as that's where the bulk of the load comes from. Compiles still work and I'm seeing hits on thefile_redirectmetric.I couldn't get acceptance tests working against this, or signed URLs working in dev. Tested in staging.
Potential Impact
Requires a settings change to proxy the files - assuming we're just making this change for readonly, this affects compiles.
Manual Testing Performed
Deployment
Requires setting
ALLOW_REDIRECTS: 'true'forfilestore-readonlyMetrics and Monitoring
There are two new metrics:
file_redirectandfile_redirect_errorso that we can watch for problems and make sure that it's doing what we want.