humanmade / S3-Uploads

The WordPress Plugin to Store Uploads on Amazon S3
1.92k stars 389 forks source link

Improve media library performance in some situations #596

Closed claudiulodro closed 1 year ago

claudiulodro commented 2 years ago

Closes #520 Closes #593

In my testing on some large sites with hundreds of GBs of images in S3, the long loading times when displaying the media library that issues report are due to the s3:// protocol in the path, (and likely other plugins inadvertently creating network requests to S3). If I've understood the logic correctly, the s3:// protocol is only needed when working with the filesystem, and displaying the media library doesn't need to access the filesystem (unless I've missed some important use-case).

Not having the s3:// protocol available during the ajax request that displays the media library dramatically improves the performance of the media library without appearing to cause any side effects in my testing.

To test:

  1. Before applying this patch, check the timing of the admin-ajax request that fetches the media library data when adding an image block or on the Media screen in WP Admin. In this test, it's 17s for that one request:

    Screen Shot 2022-08-12 at 12 33 56 PM
  2. Apply this patch, refresh the page. Verify it takes much less time for the admin-ajax request that fetches the media library data and the images still display nicely:

    Screen Shot 2022-08-12 at 12 34 22 PM
  3. Test uploading an image. Images should still upload nicely to S3.

claudiulodro commented 2 years ago

Thinking on this further, I wonder if it might be even better to only have s3:// available in the context of media upload. It's also not needed on e.g. the site frontend.

joehoyle commented 1 year ago

Hi @claudiulodro thanks for the PR, however I don't think this is the right solution. If you're seeing slowing requests to the media library because of the s3:// protocol, it likley means you have something in the code that is trying to read the s3:// files as part of that request (which default WordPress will not do). For example, maybe there's something trying to read the image files to calculate sizes in a plugin, or similar.

joehoyle commented 1 year ago

As this is old I'm going to close, but please to update us if it seems like the above explanation doesn't make sense.