search

sveltia / sveltia-cms

Alternative to Netlify/Decap CMS. Fast, lightweight, Git-based headless CMS. Modern UX, first-class i18n support, open source & free. Made with Svelte.
MIT License
901 stars 45 forks source link

Seemingly arbitrary maximum limit of entries using local git repository #224

Open xelivous opened 1 week ago

xelivous commented 1 week ago

I'm noticing some strange behaviour with the local git repository. I currently have 6 collections with the following quantities (161, 35, 1, 2, 2, 157) for a total of 358 items.

If I create a new item and bring the total up to 359, I can browse through the site and edit any information as normal. However if I refresh the page (or simply add in another collection with more than 1 entry), I will get an "unexpected error". Looking at the console log provides the following stack trace:

image

As soon as I remove any item from any of my collections (or comment out a collection) to bring the total down to 358 or lower I am able to enter the CMS once more.

kyoshino commented 1 week ago

Thanks for the heads-up. The CMS itself doesn’t set any limitation, but the error message comes from Chrome. I’ll figure out what’s going on.

kyoshino commented 1 week ago

I have duplicated a collection on my client instance, bringing the total number of entries to 1,000+, but it still works 🤔

Do you have many images, or some large files?

Perhaps it’s a Linux-specific issue. I’m on macOS. Will create a Linux VM to test.

xelivous commented 1 week ago

The 161 collection have 1x image, average around 250kb size. The 35 collection all have 1x small image each (<30kb). The other collections don't have any images. "All assets" reports 224 images.

I'll try to create some random test configs of my own to see if i can replicate it some other way.

After doing some testing, it might be some kind of random memory limit? I had 4GB of memory left on my computer but maybe chrome was running into some limit before that based off of the testing below.

  1. My indexed DB was fine at 1.1MB
  2. I browsed some thumbnails in the asset library and the indexedDB went up to 1.7MB
  3. refreshed the page and it failed to load
  4. Cleared the indexedDB, disabled cache
  5. Refreshed, Loaded fine again, refreshed, loaded fine
  6. Browsed some thumbnails to raise it above the limit, and it failed to load again.
  7. Fully closed chrome and everything else, to free up as much ram as possible on my system
  8. Reloaded up sveltia, browse through thumbnails, indexedDB up to 7MB, refresh, works fine
  9. Wait a few minutes, refresh, fails once (same tracelog as original error), refresh again, works fine
  10. The more that I refresh now, the more likely it is to fail. After about 20 refresh cycles it's now failing 100% of the time. If I restart chrome it will likely start working again.

Could be a weird bug with the local storage api.

kyoshino commented 1 week ago

Thanks for your investigation! I’ll create a bunch of images and try again.

kyoshino commented 1 week ago

I have generated 500 images in the asset library but these are still loaded (though It takes a sec; it’s a separate issue)

I think URL.createObjectURL() used for displaying thumbnails is causing a memory leak because Sveltia CMS is not revoking these URLs properly. I’ll fix the issue.

kyoshino commented 1 week ago

I’ve made the changes in v0.40.2. Do you see any difference?

xelivous commented 1 week ago

Yes, still the same issue after ensuring to pin it to 0.40.2

kyoshino commented 1 week ago

Sorry to hear that. I’ll set up a Linux VM with Parallels.

kyoshino commented 1 week ago

I have tested with Linux Mint but still can’t reproduce the problem. Will do more research.