search

project-lux / lux-webcache

Varnish cache in front of the middle tier
Other
1 stars 0 forks source link

Examine discrepancy between CloudFront and Web Cache #1

Closed gigamorph closed 2 weeks ago

gigamorph commented 3 months ago

Problem Description: From time to time, an error response (e.g., with the HTTP status code 500) is cached with CloudFront, when it shouldn't. Web cache (Varnish) doesn't cache those (verified each time the problem occurred), so there must be a problem either in the CloudFront configuration or interaction between CloudFront and the web cache.

Expected Behavior/Solution: CloudFront shouldn't cache anything (normal or error responses). Although we may decide to cache some, such as 408, that caching should happen with the Varnish web cache, not CloudFront.

gigamorph commented 3 months ago

Created a copy of the LUX environment for testing with mock responses and configuration.

Frontend URL: https://lux-front-exp.collections.yale.edu

gigamorph commented 3 months ago

Restarting the web cache often fixes the problem. While strange, it may be a clue.

gigamorph commented 2 months ago

The custom code I wrote for Varnish had a bug that caused all the error responses to be cached. My initial assertion that web cache wasn't caching those was wrong, because I was using a different URL (i.e. the AWS given URL of the web cache) to verify my claim, which, at the time I ran it, returned 200 reflecting the fixed backend, while the frontend URL (e.g. lux.collections.yale.edu/blah) was still returning the cached 500. I was practically ignoring the fact that there were different cache values for different keys.

The code is now fixed and deployed to all envs except GREEN (the current PRD).

gigamorph commented 1 month ago

It's been a while and we haven't seen any report of the same problem. I propose closing this issue.

roamye commented 2 weeks ago

Approved by UAT to close