Performance issue with media-heavy project

[timotheegoguely]

Hi,

I'm working on a Hugo project with a lot of media (~2000 files, mostly images and some PDFs) stored in /static/media and I'm using GitLab as a git provider and Static CMS v2.2.0.

When I want to display a single post from a big posts collection (~500 items), why does the app need to fetch all the repo images before being able to display the editor UI? Wouldn't there be a way to avoid all these requests and improve app performance?

As it is, it becomes almost impossible for content editors to use Static CMS to update their website content because of the slowness of the app (each page sometimes takes more than 1 minute to load).

[KaneFreeman]

Yeah this behavior is not desirable. It ideally would:

1. Wait to load only the first few image when you actually open up the media library
2. Load in the other images as necessary.

[timotheegoguely]

Is it something that can be improved easily or does it need a lot of rework?

[KaneFreeman]

Actually I'm double checking, that is what the media library is suppose to be doing already. So I need some more information to look into this further:

1. Are you storing your images at the root media_folder or are they in a collection specific folder?
2. Which page are you loading when you see all the requests? The main collection view or the editor itself?

[timotheegoguely]

1. All media (images and files) are stored at the root media_folder
2. The massive requests occurs during the editor opening after clicking on any page from any collection.

[denyskon]

Actually I'm double checking, that is what the media library is suppose to be doing already. So I need some more information to look into this further:

1. Are you storing your images at the root media_folder or are they in a collection specific folder?
2. Which page are you loading when you see all the requests? The main collection view or the editor itself?

At some point, media is loaded and saved in the entry state (mediaPaths entry property / state.mediaLibrary) which I already noticed when implementing folder support. I didn't manage to find out when that happens though

[KaneFreeman]

A list of them is loaded into the entry but its only the meta data. The actual file is not loaded until its needed for display.

I've been trying without success to replicate this. I have tested this using both Gitlab and Github backends. I've uploaded hundreds of images to them and the only ones I see load on editor load are the ones selected in image or file widgets. The rest do no load till the media library modal is opened, even then they are only loaded when you scroll to them.

@timotheegoguely, I have a hunch where this might be occurring. But I need you to help me confirm a couple of things.

1. If you go into the editor, let it finish loading, then reload the page. Does it request all of the image files?
2. If you open up Static CMS and let it sit on the collection page. Does it request a bunch of image files?

[denyskon]

@KaneFreeman For me, it seems to be caused by retrieveLocalBackup....

[KaneFreeman]

After a lot of testing, I was able to narrow down some issues that are causing these images to load. As it is today, the card/grid view in the collections page tries to load the image for the first image field it finds. It then displays that image when you view the card/grid view.

However, it was doing a couple of things that are the likely culprits of your performance issues:

1. It would load these images even when you had not switched to the card/grid view (so they were always loaded)
2. It would load images for EVERY entry in a collection

So if you had 500 entries each with a unique image, then 500 images were loaded every time you went to that collection. If you then quickly clicked into an entry it would still be loading those images and would cause the editor to sit on the loading screen until all the images had been loaded in.

The next release has some changes that will drastically help reduce this:

1. Images are no longer loaded if you are on the table view (so if you never switch to the card/grid view, then no images will be loaded at all)
2. Both the table view and card/grid view are virtualized. This means only the rows (for table view) or cards (for card/grid view) that are currently on screen (or just above/below your scroll position) will actually be rendered. Thus images for the card/grid view will only be loaded when you switch to that view and actually scroll to a given entry.

Since the table view is the default, this means:

• 0 images will be loaded on initially going to a collection
• If you switch to the card/grid view, the initial load, which will vary based on the size of your monitor, should be around or less than 20 images regardless of the total entry count.

This fix also completes #36

[timotheegoguely]

Fantastic! Looking forward to testing the next release!