Development server LiveReloads before change is ready

jneidel commented 6 months ago

In my workflow the live reload happens to early (before hugo has finished building the change), so that I have to manually reload to see the rendering of my change.

The server command is: hugo server -D --navigateToChanged

Broken flow

I do a change in vim
I switch to my browser (leaving the window makes vim write the file)
Live reload is performed in the browser
Hugo finishes building the change
I have to manually reload to see the change

Working flow

I do a change in vim
I write the change to file
I let hugo finish building the change
I switch to my browser
Live reload is performed in the browser and I see my change

notes

Things I tried that don't effect the behavior:

--disableLiveReload
--poll 700ms
--renderToMemory

Because of navigateToChanged it will navigate to the right page, but the change will not be there yet.

Here are some rebuild times:

Change detected, rebuilding site (#1).
2024-05-05 22:29:48.715 +0200
Source changed /dev/_index.md
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Total in 261 ms

Change detected, rebuilding site (#2).
2024-05-05 22:29:56.715 +0200
Source changed /dev/_index.md
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Total in 351 ms

Change detected, rebuilding site (#3).
2024-05-05 22:30:01.715 +0200
Source changed /dev/_index.md
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Total in 375 ms

Change detected, rebuilding site (#4).
2024-05-05 22:30:11.216 +0200
Source changed /dev/_index.md
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Total in 200 ms

Change detected, rebuilding site (#5).
2024-05-05 22:30:21.716 +0200
Source changed /dev/_index.md
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Total in 338 ms

I switched my theme to congo, I do not remember this being an using with ananke (maybe less heavy.)

What version of Hugo are you using (`hugo version`)?

$ hugo version
hugo v0.125.5+extended linux/amd64 BuildDate=unknown

jmooring commented 6 months ago

I am unable to reproduce this with v0.125.6 (I did not test v0.125.5). My test site has 10 pages, and each page renders a side bar containing the titles of all pages in the section. I tested your sequence using both vim (version 8.2.2121) and VS Code.

hugo v0.125.6-69ede10edcd539380914bbee58d4d32953dd8b43+extended linux/amd64 BuildDate=2024-05-05T10:52:52Z VendorInfo=gohugoio

I believe it is happening, I just can't reproduce the problem.

jneidel commented 6 months ago

My site has a few more pages and images to process:

                   | EN  | DE  
-------------------+-----+-----                                            
  Pages            |  53 | 18          
  Paginator pages  |   1 |  0  
  Non-page files   |  70 | 38  
  Static files     |   7 |  7                
  Processed images | 338 |  0  
  Aliases          |  16 |  5    
  Cleaned          |   0 |  0

Which means longer build time. Also my switching is very fast (saving on loss of focus + i3 adds no delay/animation to showing the browser).

I think the issue is a result of the build (started on write) taking longer than it takes my browser to gain focus (and reload.)

I can reproduce it on the latest version:

hugo v0.126.0-DEV-c46d603a027a715db604d39f7a2e68a157ef0001+extended linux/amd64 BuildDate=2024-05-05T11:05:28Z

Vim config to write on losing focus: au FocusLost,WinLeave * :silent! w!

jmooring commented 6 months ago

Hugo's debug log upon saving a changed file...

editor = gedit

DEBUG Received System Events: [
    CHMOD         "/home/jmooring/code/hugo-testing/content/posts/.goutputstream-4IYFN2"
    RENAME        "/home/jmooring/code/hugo-testing/content/posts/.goutputstream-4IYFN2"
    CREATE        "/home/jmooring/code/hugo-testing/content/posts/post-1.md"
]

edit = vim

DEBUG Received System Events: [
    WRITE         "/home/jmooring/code/hugo-testing/content/posts/.post-1.md.swp"
]
DEBUG Received System Events: [
    WRITE         "/home/jmooring/code/hugo-testing/content/posts/.post-1.md.swp"
]
DEBUG Received System Events: [
    CHMOD         "/home/jmooring/code/hugo-testing/content/posts/4913" REMOVE        "/home/jmooring/code/hugo-testing/content/posts/4913"
    RENAME        "/home/jmooring/code/hugo-testing/content/posts/post-1.md"
    WRITE         "/home/jmooring/code/hugo-testing/content/posts/.post-1.md.swp"
    CREATE        "/home/jmooring/code/hugo-testing/content/posts/post-1.md" WRITE         "/home/jmooring/code/hugo-testing/content/posts/post-1.md"
    WRITE         "/home/jmooring/code/hugo-testing/content/posts/post-1.md"
    CHMOD         "/home/jmooring/code/hugo-testing/content/posts/post-1.md" WRITE         "/home/jmooring/code/hugo-testing/content/posts/.post-1.md.swp"
    REMOVE        "/home/jmooring/code/hugo-testing/content/posts/post-1.md~"
]

From the above, it's apparent that vim is doing a lot of things when editing and saving a file.

Can you repeat your test with another editor, perhaps gedit or vscode?

jneidel commented 6 months ago

It reproduces in emacs.

If the file is very short it does not happen. Otherwise I can reliably reproduce it in both editors.

Feel free to try it with my site yourself: jneidel/blog (e.g. transcoding page)

This is the debug log of one reproduction:

Change detected, rebuilding site (#62).
2024-05-08 07:46:48.445 +0200
DEBUG cachebuster: Matching "content/_index.md" with source "(postcss|tailwind)\\.config\\.js": no match
Source changed /_index.md
DEBUG Direct dependencies of "/" (*hugolib.pageState page-112) =>
          __anonymous
          shortcodes/hovericon.html
          /guide/transcoding/transcoding-example-jess-crf-26.png
          /icons/email.svg
          /tags/no-complaints
          /talk/tech-problems
          /img/profile-picture.jpg
          /guide/transcoding/transcoding-reason.png
          /guide/transcoding/jellyfin-video-codecs.png
          /misc/reflection/2023-44
          /guide/transcoding/torrent-source-webrip.png
          partials/picture.html
          _default/_markup/render-image.html
          shortcodes/figure.html
          shortcodes/button.html
          partials/inline/h-rh-l/validate-fragment.html
          /guide/transcoding/transcoding-example-seinfeld.png
          shortcodes/icon.html
          /guide/transcoding/torrent-source-web-dl.png
          shortcodes/lead.html
          /icons/rss.svg
          /guide/transcoding/transcoding-example-jess-crf-38.png
          _default/_markup/render-link.html
          /guide/transcoding/transcoding-example-office.png
          /guide/transcoding/transcoding-cpu-at-80.png
          /guide
          _default/_markup/render-heading.html
          /guide/transcoding/jellyfin-audio-codecs.png
          /icons/gpg.svg
INFO  build:  step process substep resolve page output change set changes 1 checked 139 matches 52 duration 2.041451ms
INFO  build:  step process substep gc dynacache duration 1.117732ms
INFO  build:  step process substep collect files 3 files_total 3 duration 397.337µs
INFO  build:  step process duration 4.028937ms
DEBUG Direct dependencies of "/dev/transmission-wireguard" (*hugolib.pageState page-98) =>
          _default/_markup/render-heading.html
INFO  build:  step assemble substep resolve page output change set changes 1 checked 67 matches 1 duration 600.862µs
INFO  build:  step assemble substep gc dynacache duration 667.185µs
INFO  build:  step assemble duration 2.375169ms
WARN  The "link" render hook was unable to resolve the destination "++" in guide/browser/index.md
DEBUG Write redirect to main language en: //localhost:1313/
INFO  build:  step render substep pages site en outputFormat html duration 61.606035ms
INFO  build:  step render substep pages site en outputFormat json duration 28.148955ms
INFO  build:  step render substep pages site en outputFormat rss duration 8.088973ms
INFO  build:  step render substep pages site de outputFormat html duration 29.623676ms
INFO  build:  step render substep pages site de outputFormat json duration 2.916619ms
INFO  build:  step render substep pages site de outputFormat rss duration 7.442455ms
INFO  build:  step render pages 20 content 45 duration 138.918243ms
INFO  build:  step postProcess duration 14.437µs
INFO  build:  duration 146.295138ms
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Total in 147 ms

jmooring commented 6 months ago

This may not be related, but emacs also creates lock files: https://github.com/gohugoio/hugo/issues/6773#issuecomment-813161237

Can you repeat your test with another editor, perhaps gedit or vscode?

jneidel commented 6 months ago

Can confirm it also happens with gedit.

jmooring commented 6 months ago

I was also unable to reproduce this with the Hugo documentation repository, which isn't small:

                   | EN   
-------------------+------
  Pages            | 755  
  Paginator pages  |  82  
  Non-page files   |  47  
  Static files     | 117  
  Processed images | 701  
  Aliases          |  56  
  Cleaned          |   0

I don't use i3, and I have no idea if that would be a factor here.

bep commented 6 months ago

Web Server is available at //localhost:1313/ (bind address 127.0.0.1)

I'm not saying this is connected to the above, but you should define a baseURL with a protocol. We currently do no validation of this (mainly because many depend on some odd setups where / is set as baseURL).

jneidel commented 6 months ago

I don't think it's so much about project size, but about the contents of the page to be regenerated.

I tried the documentation repo and can't reproduce it there.

It might happen because there are images being reprocessed with hugo pipes (my theme does this.) This is the case on the pages in my project that I could reproduce it on.

You can try in on my project:

git submodule init
git submodule update
hugo server -p 1313 --navigateToChanged

And then open http://localhost:1313/guide/transcoding/

bep commented 6 months ago

And then open http://localhost:1313/guide/transcoding/

I tested your project (good looking site) and did misc edits in different Markdown files, and all changes refreshed in the browser (build times 30-60ms).

Could you try hugo server -p 1313 --navigateToChanged --renderToMemory and see if it behaves differently?

What I can say is:

We do wait for the build to complete before we reload.
We do, however, depend on the OS to flush any changes to disk after we close the file. In theory, this could be a culprit, but I have never seen it myself.

jneidel commented 6 months ago

I had already tried --renderToMemory and I tried it again and can still reproduce.

My build time is significantly slower at 90-200ms.

Maybe a video recording helps?

OS:

                   -`                    jneidel@jneidel-x270 
                  .o+`                   -------------------- 
                 `ooo/                   OS: Arch Linux 
                `+oooo:                  Host: 20HMS12KDE ThinkPad X270 
               `+oooooo:                 Kernel: 6.8.8-arch1-1 
               -+oooooo+:                Uptime: 3d 8h 12m 
             `/:-:++oooo+:               Packages: 1179 
            `/++++/+++++++:              Shell: zsh 
           `/++++++++++++++:             Resolution: 1920x1080 
          `/+++ooooooooooooo/`           WM: i3 
         ./ooosssso++osssssso+`          Terminal: tmux 
        .oossssso-````/ossssss+`         CPU: i5-7300U (4) @ 3.500GHz 
       -osssssso.      :ssssssso.        GPU: HD Graphics 620 
      :osssssss/        osssso+++.       Memory: 4509MiB / 15645MiB 
     /ossssssss/        +ssssooo/-
   `/ossssso+/:-        -:/+osssso+-                             
  `+sso+:-`                 `.-/+oso:                            
 `++:.                           `-/+/
 .`                                 `/

gohugoio / hugo