Solves #51 where using a long-lived cache file on the local tmp filesystem often ends up with poisoned/broken caches due (I think) to a combination of two factors:
Not running the cache expiry logic (so even though the cached data should only live for 72hrs it actually lives forever)
Uploading a file to S3 can return a positive response from the S3 API before S3 is ready to serve that file.
The crux of this issue, as far as I could tell, appears to be that S3 will tell the CMS that the file was uploaded successfully before it is ready to serve it. The CMS then immediately fans out a few requests (e.g. it tries to download the file straight away to render a CMS thumbnail, and in a separate request it tries to download the file to render a larger preview thumbnail). This leads to servers then caching an HTTP 404 for that file, because the S3 API isn't aware of the file that was just uploaded yet.
This solves the issue by using an in-memory cache. The underlying issue may still exist, but isn't exposed because we don't cache the failed API responses beyond a single HTTP call (and anecdotally at least, the Memory adapter seems to not cache results where the API fails, leading to a second call that often succeeds).
I would suggest we release this as part of a v2.0.0 of the module as it's technically, I think at least, a breaking YML config change.
Note that this issue only seems to affect sites that run on multiple servers e.g. as part of an ECS cluster or some other multi-server architecture. When it's just a single server, the HTTP requests get queued up enough that they don't tend to fail (at least, this is my theory, it's hard to know in reality).
Solves #51 where using a long-lived cache file on the local tmp filesystem often ends up with poisoned/broken caches due (I think) to a combination of two factors:
The crux of this issue, as far as I could tell, appears to be that S3 will tell the CMS that the file was uploaded successfully before it is ready to serve it. The CMS then immediately fans out a few requests (e.g. it tries to download the file straight away to render a CMS thumbnail, and in a separate request it tries to download the file to render a larger preview thumbnail). This leads to servers then caching an HTTP 404 for that file, because the S3 API isn't aware of the file that was just uploaded yet.
This solves the issue by using an in-memory cache. The underlying issue may still exist, but isn't exposed because we don't cache the failed API responses beyond a single HTTP call (and anecdotally at least, the
Memoryadapter seems to not cache results where the API fails, leading to a second call that often succeeds).I would suggest we release this as part of a v2.0.0 of the module as it's technically, I think at least, a breaking YML config change.
Note that this issue only seems to affect sites that run on multiple servers e.g. as part of an ECS cluster or some other multi-server architecture. When it's just a single server, the HTTP requests get queued up enough that they don't tend to fail (at least, this is my theory, it's hard to know in reality).