Open toorog opened 4 months ago
I created a test case in 74e9129568e3b506a34205f11d024400f833a907 -- assuming it would fail, but it doesn't.
Which means that we would need more information to figure out what happens (e.g. the full site source).
A general note:
I meanwhile stripped my page down to a minimal example that already shows an increase from ~100ms to ~1s rebuild time for 20 (empty) pages when using 0.123.0 or newer.
This is the setup I use:
hugo.yaml
baseURL: 'https://example.com'
languageCode: 'en-us'
title: 'ASCIIDoc Performance Test'
theme: hugo-geekdoc
watch: true
security:
exec:
osEnv:
- (?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM|GEM_HOME)$
allow:
- "^python$"
- "^asciidoctor$"
timeout: 120s
taxonomies:
tag: tags
category: categories
enableGitInfo: true
markup:
asciidocExt:
attributes: {}
backend: html5
extensions: [
"asciidoctor-diagram"
]
failureLevel: fatal
noHeaderOrFooter: true
preserveTOC: true
safeMode: unsafe
sectionNumbers: false
trace: false
verbose: false
workingFolderCurrent: true
content/Page<n>.adoc
(I copied this file 10 times and replaced <n>
with an actual number)
---
title: Page <n>
---
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec auctor, nunc nec
include::../include.adoc[]
_index.adoc
---
title: Mainpage
---
[pass]
{{< toc-tree >}}
include.adoc
This is some included text
Also downloaded the Geekdoc template in version 0.45.0 and extracted it to themes/hugo-geekdoc
@toorog I have the similar issue in #12095 (seems to be the same issue). Have you tried running hugo with --logLevel debug
̇? In this case it dumps some information about cachebuster and showing some page dependencies that needs to be rebuilt. In my case I see a lot of unrelated files rebuilding on each edit. Do you see the same?
This is a minimal example matching @toorog's description:
git clone --single-branch -b hugo-github-issue-12375 https://github.com/jmooring/hugo-testing hugo-github-issue-12375
cd hugo-github-issue-12375
hugo server
The Geekdocs theme has several menu partials that walk the content tree, so I'm not surprised that a change in one place requires everything to be re-rendered. I haven't spent any time looking at whether the partial calls are cached, or if that would help the situation. AsciiDoc rendering via Hugo is very slow compared to Markdown rendering, which amplifies the difference in rebuild times when comparing versions.
I ran some benchmarks for the following versions:
The build used HUGO_MEMORYLIMIT=58
and --templateMetrics --templateMetricsHints --minify --logLevel debug --gc
(with cpu/mem profiles), after the initial build the output directory was removed (rm -rf public
), then the command was ran again for the rebuild.
Between different versions the cache was cleared (rm -rf ~/.cache/hugo_cache resources public
).
Total build times:
log_0.121.0_initial: Total in 532686 ms
log_0.121.0_rebuild: Total in 213669 ms
log_0.121.2_initial: Total in 529977 ms
log_0.121.2_rebuild: Total in 213240 ms
log_0.122.0_initial: Total in 1975808 ms
log_0.122.0_rebuild: Total in 213441 ms
log_0.123.8_initial: Total in 902164 ms
log_0.123.8_rebuild: Total in 885394 ms
log_0.124.1_initial: Total in 945685 ms
log_0.124.1_rebuild: Total in 885296 ms
log_0.125.7_initial: Total in 937988 ms
log_0.125.7_rebuild: Total in 929117 ms
step duration for 0.122.0 / 0.123.8:
log_0.122.0_rebuild: INFO build: running step "process" duration "1.744710313s"
log_0.122.0_rebuild: INFO build: running step "assemble" duration "14.657683396s"
log_0.122.0_rebuild: INFO build: running step "render" duration "3m10.764673593s"
log_0.122.0_rebuild: INFO build: running step "postProcess" duration "4.720154797s"
log_0.123.8_rebuild: INFO static: syncing static files to / duration 11.952502ms
log_0.123.8_rebuild: INFO build: step process substep collect files 12000 files_total 12000 duration 3.370278353s
log_0.123.8_rebuild: INFO build: step process substep collect files 5456 files_total 17456 duration 1.250082632s
log_0.123.8_rebuild: INFO build: step process duration 4.620741498s
log_0.123.8_rebuild: INFO build: step assemble duration 551.907366ms
log_0.123.8_rebuild: INFO build: step render substep pages site en outputFormat html duration 10m34.505125734s
log_0.123.8_rebuild: INFO build: step render substep pages site en outputFormat json duration 5.027015038s
log_0.123.8_rebuild: INFO build: step render substep pages site en outputFormat rss duration 3m54.393765859s
log_0.123.8_rebuild: INFO build: step render pages 17450 content 24210 duration 14m33.973647923s
log_0.123.8_rebuild: INFO build: step postProcess duration 4.878336778s
log_0.123.8_rebuild: INFO build: duration 14m44.035089671s
From 0.123.x on, the rebuild time almost tripled and got as slow as the initial build (using HTML as output format).
cpu-0.122.0_rebuild: https://flamegraph.com/share/ca5618fc-12e8-11ef-8204-e6b38c1ccd74 cpu-0.123.8_rebuild: https://flamegraph.com/share/bf969a9c-12e8-11ef-a542-36adf59c1176
As only the cache directories are preserved and public
is deleted between the initial build and rebuild for each version, there might be another regression affecting performance?
I am building a page containing ~250 asciidoc files. Using the same content + config, these are the re-build times when touching a file while
hugo server
is running:v0.122.0
: ~1.4sv0.123.0
and newer: >40sFor
>0.123.0
the debug logging shows that on every change, all pages are being rebuild. Whereas with0.122.0
and before, only the changed page was rebuild. This might be related to the following changelog entry:However, I was not able to dig any deeper since there's no commit or issue linked.
What version of Hugo are you using (
hugo version
)?Does this issue reproduce with the latest release?
Yes