praiskup
commented
1 year ago Does it mean that this needs to be fixed in AWS CloudFront, or can we fix it by configuration?
Open praiskup opened 1 year ago
praiskup
commented
1 year ago Does it mean that this needs to be fixed in AWS CloudFront, or can we fix it by configuration?
gstrauss
commented
1 year ago https://github.com/fedora-copr/copr/issues/2011 dates back to https://pagure.io/copr/copr/issue/2001 over 16 months ago and contains numerous suggestions for improving the service, along with step-wise incremental code steps to do so.
Original issue: https://bugzilla.redhat.com/show_bug.cgi?id=2084511 Opened: 2022-05-12 10:22:58 Opened by: Lev Veyde
Lev Veyde commented at 2022-05-12 10:22:58:
Description of problem:
It was incidentally discovered that if a request is made, with Python requests package, for the build directory before the builds completes, then afterwards the server will continue to return only that cached version (for up to 24 hours).
Further debugging showed that only affected fetching the information from that particular host, and only when using the Python's requests library - requesting the same URL on the same host using other tools i.e. wget or curl returns the correct, up to date file.
Some further debugging showed that the issue only appears when the 'Accept-Encoding: gzip, deflate' header is added to the request i.e. :
curl -L -v https://download.copr.fedorainfracloud.org/results/ovirt/ovirt-master-snapshot/centos-stream-8-x86_64/04395856-ovirt-engine -H 'Accept-Encoding: gzip, deflate'
would return the stale version of the page, while this:
curl -L -v https://download.copr.fedorainfracloud.org/results/ovirt/ovirt-master-snapshot/centos-stream-8-x86_64/04395856-ovirt-engine
would return an up-to date one.
Trying to add cache control headers to the request had no effect, i.e. :
would return the old cached copy.
Not sure if the issue is due to some configuration on the Copr side, or it's an issue with Amazon CloudFront service, that improperly handles the case of the compressed cached results.
Also not sure at this stage if the result will be returned just for the same host that made the request, or everyone who falls on that particular server that will serve the cached result from the AWS's CloudFront.
Version-Release number of selected component (if applicable):
How reproducible: not sure at this stage
Steps to Reproduce:
check what you get in data.headers and data.text
Actual results: The returned page will be in an out of date state
Expected results: The returned page should be updated one, just as an uncompressed one.
Additional info:
Miroslav Suchý commented at 2022-05-12 11:03:58:
seems like https://stackoverflow.com/questions/18774069/amazon-cloudfront-cache-control-no-cache-header-has-no-effect-after-24-hours but I do not see way how to configure ttl for cloudfront
Lev Veyde commented at 2022-05-12 11:10:03:
(In reply to Miroslav Suchý from comment #1)
Only in this case the issue is only with the compressed data, thought for complete test we'll need to test making requests both compressed and uncompressed, and see if there is any difference in handling.