search

jackyzha0 / quartz

🌱 a fast, batteries-included static-site generator that transforms Markdown content into fully functional websites
https://quartz.jzhao.xyz
MIT License
6.87k stars 2.48k forks source link

Site hanging when using >8000 pages #1492

Open franz101 opened 2 days ago

franz101 commented 2 days ago

First of all I LOOOVE QUARTZ thank you (just started sponsoring)

Describe the bug When using more than >8000 pages page performance review from google page speeds takes 25 seconds for the main thread To Reproduce Steps to reproduce the behavior:

  1. Go to https://beat-pool.think-complete.com
  2. Check performance on: https://pagespeed.web.dev/analysis/https-beat-pool-think-complete-com/qdwqqrtqqe?form_factor=desktop
  3. Expected behavior Good page speed performance Results: Minimize main-thread work 26.2 s

Screenshots and Source If applicable, add screenshots to help explain your problem.

Screenshot 2024-10-08 at 11 44 55

Source code: https://github.com/franz101/beat-pool

saberzero1 commented 2 days ago

I noticed you have a Python script to preprocess all files, adding dozens of links to thousands of files. You seem to be hitting the file size capacity of GitHub as well.

Anyway, I suspect the performance costs mostly result from the explorer tree. We should probably not load that on mobile/fetch it only on larger screens on when screen sizes change, as it is not visible on mobile by default anyway.

Can you remove the Explorer() component from your quartz.layout.ts file and check if the performance improves significantly? If it does, we could explorer lazy loading the explorer. Otherwise I have to do some more investigation.

franz101 commented 2 days ago

@saberzero1 yeah that is probably the main reason, it maybe neesd max rows + lazy loading when scrolling? search is working very fast.

If you download this page for example: https://beat-pool.think-complete.com/producers/Broke-Boys

you will see it's 1.2 mb. Though I don't why this would be an issue.

franz101 commented 2 days ago

so basically im linking between artists. With 8000 artists * and each page through the nav being 1.3mb we end up with 20gigs uncompressed :D

I was able to make it work gzipping the files and using transform in cloudflare r2. can provide a guide at some point for this. but in general building takes 5 minutes at the moment

saberzero1 commented 2 days ago

Searching and popovers (previews when hovering over links) are fetched when needed. Explorer is not at the moment. We should probably change that.

Did you test the performance without explorer component?

franz101 commented 2 days ago

I tried obsidian and it worked, but it misses the customizability I love from quartz

franz101 commented 2 days ago

Without explorer building is 2x faster (2 minutes)

saberzero1 commented 2 days ago

Without explorer building is 2x faster (2 minutes)

I don't mean build performance. I mean pagespeed performance.

franz101 commented 2 days ago

pagespeed also fine then, another question here: they layout is rendered twice? for mobile and desktop

franz101 commented 2 days ago

you have been super helpful. for a custom navbar can I provide nav.md or do I need to write a custom explorer logic?

saberzero1 commented 2 days ago

pagespeed also fine then, another question here: they layout is rendered twice? for mobile and desktop

It partly used to be. Not anymore. We currently use a grid-based layout. This allows us to change the layout based on screensize without duplicate components.

We should probably change the Explorer component logic to fetch folder content only when requested (a.k.a. opened). You have about 14.000 links in your Explorer. Even after lazy loading, you might want to break up those in several subfolders. Otherwise your 25 second wait is during folder toggle.

I'll take a look tomorrow.