Ability to use Redis (wp_cache) to store recived data

[sebastianthulin]

Issue Description

Hello,

I am seeking information about the possibility of caching requests for Swift Object Storage. My website is currently experiencing performance issues, and I have observed considerable traffic directed towards Swift. I believe caching these requests could potentially enhance performance.

Goal

My aim is to create a short-lived cache to reduce the number of external requests, particularly for verifying the existence of images (?) from the backend.

Questions

• Are there any available filters or methods to cache requests to Swift Object Storage?
• What are the best practices for implementing such a caching mechanism?
• Are there any existing cache of this, that may be misconfigured? What are the steps already taken to minimize external (towards object cache) requests?

Steps to Reproduce

Host a website that frequently interacts with Swift Object Storage. Monitor the site's performance and traffic patterns (used newrelic). Notice the impact of frequent requests to Swift on overall site performance.

Behavior

High traffic to Swift Object Storage is contributing to performance degradation on the site. I have a RPM ( request per minute ) averaging at 160, the second largest rpm (of antother service) is making 3 rpm in the same timeframe.

Additional Information

Any guidance or suggestions for caching strategies, available filters, or best practices would be greatly appreciated. Thank you!

[rmccue]

I'm not sure what Swift Object Storage is, but presumably it's an S3-compatible external API?

I'm not exactly clear what you're trying to achieve here; are you purely aiming to avoid requests to the external API for performance?

There's no plans to implement additional object caching in S3 Uploads, as WordPress already has various caching layers which store derived data and avoid file access where possible (e.g. pregenerating and caching EXIF data, caching size data, etc). Off the top of my head, I can't think of any additional pieces which would benefit from more object caching.

[sebastianthulin]

Hi!

The Swift object store in OpenStack serves as an S3 replacement. It essentially supports the core API, though it lacks some advanced features. However, I haven't noticed any integration issues due to these missing features, so it can be considered a viable alternative to AWS S3.

Our main goal is to avoid unnecessary requests, as this can save about 80-400ms depending on the number of images on a page. While caching these requests might seem counterproductive, I believe I read in the documentation that frontend performance shouldn't be affected by this plugin. However, in our environment, it does seem to have an impact.

We use wp_get_attachment_image_src() in the affected requests. Would this trigger a backend callback to the "bucket" in a standard installation? We have a comprehensive system for "on-the-fly" image resizing, which might interfere with normal behavior.

• Are any callbacks to the bucket expected normally?
• Are there any functions responsible for these calls that I can trace?

I use New Relic to gather my conclusions on this matter.

[joehoyle]

We use wp_get_attachment_image_src() in the affected requests. Would this trigger a backend callback to the "bucket" in a standard installation? We have a comprehensive system for "on-the-fly" image resizing, which might interfere with normal behavior.

It should not no, and if you are seeing this, it's likely the result of a bad acting plugin calling things like filesize() etc on the attachment file. WordPress may also do that if you attachment_metadata is not correctly populated.

This is why we don't have a short-lived cache in S3 Uploads, if things are written correctly there should be no need to read the file from S3 (or Switch Object Store in your case) on any critical rendering / performance sensitive places.

[sebastianthulin]

Would this also include a file_exists function? I might need to check if the file exists before creating a resize in my own code. If so, I'll focus on this functionality. Thanks!

To provide some context, here is the functionality that may be causing the issue based on the analysis: https://github.com/helsingborg-stad/Municipio/blob/main/library/Theme/OnTheFlyImages.php#L440

[joehoyle]

Yes it would include file_exists. If you are doing on the fly resizing that stores the resized thumbnails in S3, then I'd recommend you also store what sizes you've generated in the database; though that's outside of the scope really of S3 Uploads. WordPress treats anything in the Uploads directory as "will be slow to read / write to" so I'd recommend using the same assumptions in your on-demand image resizing.

Let me also take the opportunity to plug another project, tachyon which is an on-demand image resizing microservice that integrates with S3 and is built to work with S3 Uploads. I don't know whether than could also be an option. Tachyon works by a URL resizing API, so no resizing is done as part of PHP / rendering pages

[sebastianthulin]

Thank you for confirming the inclusion of file_exists. I appreciate the advice on storing the generated sizes in the database; I was thinking along the same lines.

Thanks for the heads-up about Tachyon! It sounds promising, and I'll definitely explore whether it could be a good fit for our needs.

Thanks again for the valuable insights!