search

gohugoio / hugo

The world’s fastest framework for building websites.
https://gohugo.io
Apache License 2.0
75.52k stars 7.51k forks source link

Add build tags for minimal CI-optimized builds #12308

Open Seirdy opened 7 months ago

Seirdy commented 7 months ago

Hugo already has a +nodeploy build tag to exclude dependencies related to deployment. Unfortunately, with over 100 dependencies, its size still exceeds 50 MiB before binary compression. This is a request to add a few additional build tags to slim down builds.

In CI, it's normal to just use Hugo to build a site. Serving a site (let alone with live-reload or TLS), renderSnippets, polling, profiling, manpage rendering, etc. are unnecessary. A CI-optimized build tag that excludes any dependencies for local development, such as +nodevelopment, could exclude any dependencies related to this.

A very large number of dependencies relate to image processing. A static site with pre-optimized images could skip dozens of dependencies with a +noimage tag.

Sites that use just one markup format could disable others: +noasciidoc, +noorgmode, etc.

I imagine that CI-optimized +nodeploy builds with limited support for alternative markup formats could be quite popular for "build and rsync" workloads.

Seirdy commented 7 months ago

I'll also add that this can reduce the disk space required to build Hugo. On my machine, Hugo dependencies take up more space in my Go mod cache than almost any other Go program. A fresh build after cleaning my mod cache is incredibly slow.

A minimal version of Hugo containing only what's necessary to build a site and exit would cut down on build times and disk space.

bep commented 7 months ago

Why not use a prebuilt binary in CI?

Seirdy commented 7 months ago

I was describing my workstation; my CI uses a tarball of binaries I build locally. Though it does take quite a while to build on my 2013 notebook.

bep commented 7 months ago

I was describing my workstation; my CI uses a tarball of binaries I build locally. Though it does take quite a while to build on my 2013 notebook.

Your not answering my question; why not use a prebuilt binary?

As to the issue itself, I don't think it's possible/practical; what is a CI-optimized build? How do we know what features CI needs?

Seirdy commented 7 months ago

I imagine that most CI uses of Hugo build sites but don't serve them; a +noserve tag may be more appropriate. I figured that if it's worth adding support for +nodeploy then it may be worth supporting +noserve.

Re. why no prebuilt binary: I often build from HEAD and have a few trivial patches. I may at one point try experimenting with things like Go's new PGO features to speed up site rebuilds.

cmahnke commented 7 months ago

Given that you already have an CI infrastructure, why not decouple building of a site from building Hugo? I suppose there is no direct dependency between you patches an the site (as this would imply questionable design choices)? Just create a CI pipeline that builds you binary and reuse it in the next pipeline building the site, this might also speed up things since you only need to rebuild Hugo after changing / reapplying patches to Hugo.

Seirdy commented 6 months ago

I feel that these arguments also apply to the existing +nodeploy tag.

If reducing dependencies, speeding up build times, and shrinking binaries aren't useful on their own then I'm not sure why Hugo supports release builds with build tags at all.

Seirdy commented 6 months ago

Right now, I rsync a tarball with binaries over to a server that my CI fetches after verifying that there are no regressions.

I'm not sure that I'd consider setting up a multi-pipeline process to continuously build a binary and automatically test for regressions and breaking changes just to build a single 200-page site a solution as much as a small long-term side-project.

I get that building Hugo from source is a bit of a niche use-case, but perhaps that's due in part to how hard it is to build. None of this would be necessary if we could just build Hugo's website compiler functionality.

cmahnke commented 6 months ago

Well, it's not hard to build at all. I would suspect that's more due to the fact that nowadays its's not as much a benefit to build things from source as it used to be (15 years ago)...

I'm not that much into Go to know what cross compiler tool chains are available, so I would tackle the problem by just using a GitHub Action using Qemu and Docker, try to build the binary there, extract and package it. It shouldn't be hard to set up. Using github would also free you from disk, (CPU) speed and network bandwidth constrains. Note: This might not be the best approach.

And since you might be the only one who uses it, it's completely up to you how much tests you do...

Links: