search

timlrx / tailwind-nextjs-starter-blog

This is a Next.js, Tailwind CSS blogging starter template. Comes out of the box configured with the latest technologies to make technical writing a breeze. Easily configurable and customizable. Perfect as a replacement to existing Jekyll and Hugo individual blogs.
https://tailwind-nextjs-starter-blog.vercel.app/
MIT License
8.83k stars 2.05k forks source link

Cant build a blog with thousands of pages #729

Open Faiiya opened 1 year ago

Faiiya commented 1 year ago

Describe the bug I have a blog with around 12500 pages, all in mdx. I have tried everything i can but it just refuses to build. most of the time i get no error, others i get javascript heap out of memory, or yarn exit code 137 so seems to be memory related, i have tried to do it on vercel or with 16gb ram but no luck. i have tried with a clean clone of this repo and just adding the blogs and it still fails.

To Reproduce just clone the repo create a big number of copies of a mdx article and build

Expected behavior i expect it to build

Screenshots

System Info (if dev / build issue):

timlrx commented 1 year ago

I guess you have tested it for a small number of mdx files and it works? I have not tested it on so many mdx files.

Faiiya commented 1 year ago

yeah with small number of files it works, but someetimes it breaks with maybe 500. the maximum i suceed building was with 4000 in vercel

ttys3 commented 1 year ago

I have almost 400+ pages, sometimes it failed to build on Netlify

timlrx commented 1 year ago

Might be worth instrumenting it to see what is taking up the most memory / build time. I suspect it is the pre-rendering done on Next.js side which makes it hard to optimize. Maybe at 1000 pages+ it might be better to consider using server side rendering.

Anyone has a publicly available repo with that many pages?

timlrx commented 1 year ago

Re-reading this again and 12500 mdx pages really scares me - how do you even manage that 😆

At that scale, you should probably consider using a database backed CMS.

Faiiya commented 1 year ago

what cms do you recomend? , i dont need any editing or special tools, just simply host them and thats it.

after they are on the cms i should be able to do ISR wich should hopefully fix my problems right?

ttys3 commented 1 year ago

image

I only have 100+ mdx:

❯ fd .mdx | wc -l 
135

and some tags.

deploy failed due to build failure.

I check local, I think it is because it uses more then 8GB memory

image