search

statamic / ssg

The official Statamic Static Site Generator
234 stars 24 forks source link

Built time is very long - + 15 minutes. #16

Closed ossedk closed 3 years ago

ossedk commented 4 years ago

Built time is very long - + 15 minutes for a fairly normal site with 100 static pages.

Any way to optimize the built time?

jackmcdade commented 4 years ago

Yeah that sounds like something isn't working right there. Does it actually finish after 15 minutes or does it eventually just time out? I'm wondering if there's a memory leak or something somewhere.

ossedk commented 4 years ago

It's the same for local, server or any other enviroment.

It completes after 20 minutes or so, - the problem is that it has to rebuilt everything.

Would be very nice if you could just built one specific page, that you changed?!

Only option would be to exclude all other pages, which seems like a bit overkill when you have +100 pages.

tao commented 4 years ago

I also have a few thousand items in a collection and it takes more than an hour to generate the static website... it seems like it's about 3-5s per page. I also tried the same content on Jigsaw static site generator it only takes about ~45s...

jasonvarga commented 3 years ago

Is the time a single page gets generated about the same time you'd see it load in the browser?

tao commented 3 years ago

I believe that yes, it does take as long as it does in the browser... however it's also a bit difficult to test because of the leaking data bug #38. The results from SSG look very different from the browser and often many pages are mixed together. So it could be processing all sorts of extra things and I wouldn't be able to do an exact comparison.

tao commented 3 years ago

Something else I could try later on is downloading my entire S3 asset bucket and testing SSG as if it was a local asset container.

jasonvarga commented 3 years ago

If a page is slow in the browser, it's going to be just as slow in the SSG. The SSG is rendering the page just like it does for the browser, but writing it to a file.

If you're able to leverage {{ cache }} tags, that should help on the SSG side, too.

(If #38 was fixed, I don't expect it to be any faster than what you get in browser, although it might be making it slower)

We're going to look at ways to improve performance soon.

jasonvarga commented 3 years ago

In #54 I've added support for your files to be generated concurrently. It speeds things up a ton.

If you want to try it out:

composer require "statamic/ssg dev-async"

then

php please ssg:generate --workers=4

but change 4 to the number of CPU cores on your machine.

tao commented 3 years ago

Great, I'll try it out!

I've also been testing the {{ cache }} tags to speed up the site build and I believe caching the nav tags helped with the speed too, but I'll follow up with more once I give this a go. It also seems like the pages with attached S3 assets might be one of the bottlenecks I need to debug a bit more.

tao commented 3 years ago

I've got this dev-async running without any problems. Seeing a list of all errors at the end would useful because they would disappear previously when the log was too long from thousands of pages.

Most of my pages generate in less than a second but the ones with assets take a few minutes sometimes, my /podcasts/rss got stuck for 40+ minutes (I just cancelled it after that).

I'm going to try my asset on DO spaces instead of S3 next, and I'll also try download all the assets to localhost and run SSG again to compare the build times.

jasonvarga commented 3 years ago

Does that podcast one take a long time on the non-static version of the site?

tao commented 3 years ago

Yes, I've just emailed you a zip file with my podcasts for you to try out.