Gatsby doesn't always refresh changes

[tu4mo]

Description

Gatsby sometimes fails to reload changed components.

Issue can be reproduced after version v2.24.9, so something changed between 2.24.9-2.24.10.

Steps to reproduce

https://github.com/tu4mo/gatsby-reload-issue

git clone git@github.com:tu4mo/gatsby-reload-issue.git
cd gatsby-reload-issue
npm install
npm start

Run (multiple times if needed) in a another terminal:

node update-fakelib.js

Expected result

The scripts rewrites 10 components with current timestamp. All changed components are hot reloaded.

Actual result

Some, all or none of the components are updated, inconsistently.

Environment

  System:
    OS: macOS 11.0.1
    CPU: (8) x64 Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 14.15.1 - ~/.nvm/versions/node/v14.15.1/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.14.8 - ~/.nvm/versions/node/v14.15.1/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 87.0.4280.88
    Edge: 87.0.664.57
    Firefox: 83.0
    Safari: 14.0.1

[vladar]

Oh, I see you've already followed my advice: https://github.com/gatsbyjs/gatsby/issues/26580#issuecomment-715282412 😄

We need some way to reproduce it to move forward. Does it happen for you on any project, including our blog starter? If not, please provide a minimal reproduction and we'll be able to take another look.

[tu4mo]

Yes, I can reproduce it with blog starter. Typing something in index.js, saving quickly twice (cmd-S), making a change, saving quickly twice; doing this a few times and you'll see that the change isn't shown on the page.

[tu4mo]

[tu4mo]

I also reversed blog starter to a commit that had a version later than v2.24.0 (v2.23.12) and I can't reproduce the issue there. Clearly something changed in the refresh mechanism after v2.24.0 and made it unreliable.

@vladar Is there anything else I can do to help with this issue getting fixed? Or am I forever stuck with v2.24.0... 😞

[vladar]

I tried to reproduce it with the latest gatsby version and also on 2.24.0 but I don't see stale results. Which makes me think that it may have something to do with your environment.

Do you have Dropbox or other similar tools installed in the same folder where your gatsby site is located? Maybe some file watchers in IDE or other tools?

[vladar]

Also, can you bisect a specific version where this problem was introduced for you? The 2.23.12 - 2.24.0 range has too many changes. We need to narrow it down.

[tu4mo]

I don't have Dropbox or any other file sharing apps using code files. In my project I have watchers that built the library (Rollup and TypeScript in watch mode).

Sorry for not being clear about the version. 2.24.9 works, 2.24.10 does not.

[vladar]

Then it sounds very similar to #26192 Have you tried upgrading to 2.24.57 as suggested here: https://github.com/gatsbyjs/gatsby/issues/26192#issuecomment-690158670 ?

[tu4mo]

Tried 2.24.57, same thing, does not reload.

[tu4mo]

It feels like there's something little different in 2.24.57. After I edited a component in my library, I did get it reload by editing a Gatsby page where the component is imported. Sometimes it works, sometimes it doesn't, sometimes the whole reloading seems to freeze, and I have to restart Gatsby.

[vladar]

After I edited a component in my library, I did get it reload by editing a Gatsby page where the component is imported. Sometimes it works, sometimes it doesn't, sometimes the whole reloading seems to freeze, and I have to restart Gatsby.

But can you reproduce the problem on the blog starter with gatsby@2.24.57? If not - then it means we are still stuck without any reproduction that we could use to troubleshoot your specific issue.

[tu4mo]

Yes, I can reproduce it everytime typing rapidly aaa, cmd-S, cmd-S, bbb, cmd-S, cmd-S, ccc, cmd-S, cmd-S, etc.

[tu4mo]

https://github.com/tu4mo/gatsby-reload-issue

git clone git@github.com:tu4mo/gatsby-reload-issue.git
cd gatsby-reload-issue
npm install
npm start

In a another terminal:

node update-fakelib.js

With v2.24.57 I rarely get to last update. Running it a few times and the page stops responding.

With v2.24.0 it works.

You might need to increase the amount of components "created" and decrease the sleep.

[vladar]

Thank you so much for reproduction! 💜

I definitely see how develop is getting stuck eventually. Also in the given version range - #25815 looks like a primary suspect.

Investigating this.

[vladar]

So one potential problem is here:

https://github.com/gatsbyjs/gatsby/blob/441263ed72c6ef7b0a8da581076bfa7e786c6b30/packages/gatsby/src/state-machines/develop/index.ts#L189-L195

We call markSourceFilesClean upon finishing recompiling JS. But all changes that occur during recompile are discarded. That doesn't explain why it doesn't recover on subsequent file changes when in waiting state.

One possible hint - somehow webpack stops calling invalid hook (even though chokidar actually emits change events for those):

success Building development bundle - 5.377s
[chokidar_change] fakelib\Component.js
[webpack.compiler.hooks.invalid] fakelib\Component.js
success onPreExtractQueries - 0.004s
success extract queries from components - 0.062s
success write out requires - 0.001s
[chokidar_change] fakelib\Component.js
success Re-building development bundle - 0.204s
[webpack.compiler.hooks.invalid] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js

So the next step is to dive deep into suspend/resume of the webpack. Just leaving some notes here to keep the context around.

[vladar]

OK. I think I've found the issue. This PR should fix it (hopefully): #28237

[vladar]

Published in gatsby@2.27.2. Can you please check if it fixes the issue for you?

[tu4mo]

Tested, and it's better as in it doesn't crash anymore. But it still fails refreshing most of the time :(

Tried editing a Badge component in my library 4 times. Here is the browser log.

1. edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 10138ms

Component didn't update.

2. edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 11960ms
[HMR] Checking for updates on the server...
[HMR] Updated modules:
[HMR]  - ../lib/dist/lib/components/Badge/Badge.js
[HMR] App is up to date.

Component updated.

3. edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 11997ms
[HMR] bundle rebuilding
[HMR] bundle rebuilt in 419ms

Component didn't update.

At that time I also noticed this in terminal, not sure if related:

warning Warning: Event "xstate.after(200)#waitingMachine.aggregatingFileChanges" was sent to stopped service "waitingMachine". This service has already reached its final state, and will not transition.
Event: {"type":"xstate.after(200)#waitingMachine.aggregatingFileChanges"}

4. edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 532ms

Component didn't update.

Gave up after that.

[tu4mo]

I also see this in the terminal 4 times:

info changed file at /Users/tu4mo/Code/styleguide/lib/components/Badge/Badge.tsx

So Gatsby seems to pick up the change, but the rebuild times are much shorter.

[tu4mo]

I also updated Gatsby in https://github.com/tu4mo/gatsby-reload-issue and I can still reproduce this emulating multiple component updates.

[tu4mo]

@vladar Maybe you could re-open this issue?

[tu4mo]

@vladar Should I create a new issue?

[vladar]

Reopened. So to recap - it doesn't hang anymore but also doesn't emit HMR updates.

[tu4mo]

Any news?

[karlhorky]

@tu4mo you mentioned in the other thread that "Maybe the issue has something to do with symlinks..." - are you also using symlinks?

Did you ever get symlinks working with hot reloading on a previous version of Gatsby?

[tu4mo]

@karlhorky They were a suspect, but I discluded them with my reproduction example.

[karlhorky]

Ok, so you've had success with symlinks with Gatsby before? Sorry, kind of off-topic, just interested - not a lot of information on the topic.

[tu4mo]

@karlhorky Yes, I use yarn workspaces with Gatsby so some packages in node_modules are symlinks to those workspaces and they work fine. And by fine I mean they work just like any other packages, but after 2.24.9 HMR fails in general.

[karlhorky]

Cool, thanks for the heads-up. For me, I was trying to symlink mdx content into pages from a location outside of the repository, but this appears to be broken with hot module replacement / hot reloading (also requires that the dependencies like @mdx-js/mdx and such are set up in that external location).

I think I will also go for a monorepo with Yarn Workspaces.

Sorry for the slight detour in the conversation!

Thanks for bringing this issue up!

[tu4mo]

I updated the issue description with the reproduction, so it doesn't get lost in the comments.

Maybe @ascorbic has some ideas to fix this, since it might have something to do with the changes in https://github.com/gatsbyjs/gatsby/commit/9c84cb7f9d301ab6745022d600cc294b791dd9d7?

[d4h0]

Unfortunately, I have the same problem.

I noticed, this problem does not happen with JavaScript/JSX files that are handled by Gatsby (or rather by the WebPack setup of Gatsby).

It only seems to happen with ReScript files (ReScript is a transpired language, similar to ResonML and TypeScript), which are compiled by an external process (not within Gatsby's WebPack setup).

I assumed, the problem was somehow that a saved file "xy.res" triggers the ReScript compiler to create or update "xy.bs.js" file within a few milliseconds (the ReScript compiler is extremely fast).

My assumption was that excluding the ReScript files from Gatsby by adding ignore: ["**/*\.res"] to the options of gatsby-source-filesystem would fix the problem. Unfortunately, it didn't.

However, ReScript has a post build hook that is executed with newly create/updated JavaScript files. From there I now execute touch $new-js-file, which seems to fix the problem, fortunately.

@tu4mo: Maybe you can implement a similar workaround until the problem is fixed?

Do you have a similar setup, where files are created/updated by an external process?

Above it seems the problem happens for you even with JavaScript/JSX files that are handled by Gatsby. It's strange that this doesn't seem to happen with my setup.

@vladar: Does this help to narrow the problem down? Before I added ReScript to my project, I didn't have any problems with the auto-reload feature. Now Gatsby is basically unusable (without my workaround, and when I program in ReScript).

[tu4mo]

I tried touch:ing the components in my repro, but the buggy behavior is still the same. It doesn't matter how the files are updated. I hope the Gatsby team prioritizes some time to fix this soon.

[d4h0]

Yes, unfortunately it turned out touch-ing the files does not fix the problem completely (but it still is a big improvement for me).

[fcisio]

I had hope that fast-refresh would solve this issue, still no luck on my end. See Fast Refresh

The last stable version for HMR is Gatbsy@2.24.0, I really hope this gets fixed soon

[tu4mo]

Possibly related https://github.com/webpack/webpack/issues/6337?

[tu4mo]

I'm just poking around blindly, I don't really know what I'm doing. But it looks like FILE_CHANGE_AGGREGATION_TIMEOUT has something to do with this?

https://github.com/gatsbyjs/gatsby/blob/41488778ddcb102a5147b7acfb9631565ede958e/packages/gatsby/src/state-machines/waiting/index.ts#L8

It's set to 200, and if the changes take longer than that, some of the changes are not picked up. I tried monkey patching the value to 2000 and it seemed to help.

@vladar, @ascorbic? Any chance you could take a look at this?

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[tu4mo]

Hot reload is still broken.

[vladar]

I've added some output to webpack Watching callback to see if it captures all changes internally and here is what I get:

filesModified: [ 'C:\\dev\\tmp\\gatsby-reload-issue\\fakelib\\Component1.js' ] Suspended:  true
filesModified: [
  'C:\\dev\\tmp\\gatsby-reload-issue\\fakelib\\Component3.js',
  'C:\\dev\\tmp\\gatsby-reload-issue\\fakelib\\Component4.js',
  'C:\\dev\\tmp\\gatsby-reload-issue\\fakelib\\Component5.js',
  'C:\\dev\\tmp\\gatsby-reload-issue\\fakelib\\Component6.js'
] Suspended:  true
filesModified: [
  'C:\\dev\\tmp\\gatsby-reload-issue\\fakelib\\Component10.js',
  'C:\\dev\\tmp\\gatsby-reload-issue\\fakelib\\Component9.js'
] Suspended:  true

As you can see Component2, Component7, Component8 are not listed here. So webpack doesn't see those changes internally and doesn't recompile them. And it looks like a bug(?) in webpack Watching with suspend and resume.

[vladar]

Basically what happens is that Gatsby calls resume to recompile aggregated changes and then instantly calls suspend to start aggregating the next batch: https://github.com/gatsbyjs/gatsby/blob/bfd8c8c41db5fea654a9b84c54bbf63db0ce3a54/packages/gatsby/src/services/recompile.ts#L23-L25

The problem is that resume is actually not instant. It is a multi-step process:

1. It pauses watching
2. It starts recompiling
3. It actually resumes watching when recompilation is done

So any changes that happen in between are not seen by webpack. Basically, it is the same problem we had in Gatsby before #28237 but in webpack itself.

At least if I get this all correctly.

[vladar]

The good news is that I've tried it with the latest webpack5 (using a branch from this PR: https://github.com/gatsbyjs/gatsby/pull/29145) and the repro worked for me almost as expected. This suggests that the issue was likely fixed in webpack5 (there could be one more minor edge case there, not sure yet).

We will publish gatsby v3 pre-release soon with this PR included so that you could try it.

[vladar]

Can you try it with gatsby@3.0.0-next.5 (which includes webpack bump)?

[tu4mo]

Tested gatsby@3.0.0-next.5 and I'm seeing improvement, but it still doesn't work as reliably as it does in v2.24.9. Still seeing it fail after a few edits. Not sure if related, but in my real project rebuilds also now takes a long time. An edit takes sometimes over 20 seconds to show in the browser. In v2.24.9 they usually take only a couple of seconds.

[vladar]

Can you still reproduce it with your original reproduction? For me, it occasionally skips last component updates, but on the next save (of any file) all changes are included in the bundle.

What you describe sounds a bit different: Still seeing it fail after a few edits. Can you elaborate on how exactly it fails for you? And does it happen with the repro?

[tu4mo]

"Fails" as in changes does not end up showing in the browser. Hard reload makes them show up.

I updated the Github repo with a lower sleep value. It seems to be easier to reproduce when the components change fast.

[wardpeet]

@tu4mo any chance you can share your project with us? It could be that we'll have to change it back to style-loader if it's too slow. You can always mail me at ward@gatsbyjs.com

[tu4mo]

@wardpeet I'm afraid I can't share my real project at the moment. I'm not importing any CSS files though, only styled-components.

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[vladar]

@tu4mo PR #30194 is a workaround for this webpack issue. I've published canary gatsby@3.1.0-alpha-hmr-fix.9+3c2ef63555 can you try it and see if it helps?

[tu4mo]

Sorry but it didn't. Still reproducible with gatsby-reload-issue and in my real project.