search

webpack / webpack-dev-server

Serves a webpack app. Updates the browser on changes. Documentation https://webpack.js.org/configuration/dev-server/.
MIT License
7.77k stars 1.43k forks source link

Invalidate route does not recognise changed files when recompiling the app #2930

Closed tiagobento closed 3 years ago

tiagobento commented 3 years ago

Code

// webpack.config.js
// additional code, remove if not needed.

Expected Behavior

Navigating to /webpack-dev-server/invalidate actually recompiles the app, independently of watchOptions configuration.

Actual Behavior

Navigating to /webpack-dev-server/invalidate does not recompile the app when watchOptions.ignored is **/*.*

For Bugs; How can we reproduce the behavior?

I'm developing a webapp that is part of a monorepo using Lerna, and I want my app to be updated when one of its dependency packages is rebuilt. So I created a script that watches for changes in dependency packages within this same monorepo and rebuilds them upon changes. Leaving the watch capabilities of webpack-dev-server turned on results in unpredictable behaviour, since many things get deleted/changed/created during the recompilation of these packages and webpack-dev-server triggers recompilation many times, often leaving the app broken, since it doesn't detect the last package rebuilt.

In order to do what I need, I have to be in full control of when webpack-dev-server rebuilds the webapp, so I added **/*.* as my watchOptions.ignored, expecting webpack-dev-server to simply not watch for changes. However, after navigating to /webpack-dev-server/invalidate, on my app, I see:

[WDS] App updated. Recompiling...
[WDS] Nothing changed.

Please help with advice on how to accomplish what I need or give me guidance to how I can help with fixing this issue within this repo. Thank you in advance!

For Features; What is the motivation and/or use-case for the feature?

N/A

alexander-akait commented 3 years ago 
[WDS] App updated. Recompiling...
[WDS] Nothing changed.

webpack does invalidation and says you nothing was changed, it is expected

tiagobento commented 3 years ago

That happens even if I change files directly on my app. Is that expected? Why are watchOptions.ignored and invalidation connected? It looks to me that watchOptions.ignored is being used as "glob to completely ignore after first compilation", and not as "glob to not watch". Am I missing something?

This is actually my attempt of continue using webpack-dev-server without having to go for a more custom solution involving webpack-dev-middleware directly.

alexander-akait commented 3 years ago

invalidate - says webpack to run new build manually (even nothing was changed) watchOptions - how to watch files

I don't understand your problem

tiagobento commented 3 years ago

@alexander-akait Thanks for the reply. Let me try to be more clear.

From my understanding, webpack-dev-server is designed to work within a single NPM package and to watch for changes on the files it used to compile the webapp it's serving using webpack. Some of those files are actually inside the node_modules directory, since naturally webapps depend on 3rd party libraries almost always.

Let's say I don't have any watchOptions configured and that my webapp depends on a 3rd party library called package1. When spinning webpack-dev-server, it will watch for changes on node_modules/package1/index.js, since this file was used to compile my app in the first place, and if I manually change this file, webpack-dev-server will be invalidated and the application will be recompiled. Correct?

When I change my DevServer configuration to have watchOptions.ignored: "**/*.*", and manually change node_modules/package1/index.js, the application will NOT recompile, since webpack-dev-server is NOT watching for changes anymore, as expected. That is correct and I don't think anything should be different until this point.

Now, my problem is: With the watchOptions.ignored configuration above, after I change node_modules/package1/index.js manually and navigate to /webpack-dev-server/invalidate, the webapp does recompile, but, the changes I made to node_modules/package1/index.js are NOT being picked up by the recompilation triggered by /webpack-dev-server/invalidate. And that's a problem for me.

Hope that's more clear now! I really appreciate the help :)

alexander-akait commented 3 years ago

Now, my problem is: With the watchOptions.ignored configuration above, after I change node_modules/package1/index.js manually and navigate to /webpack-dev-server/invalidate, the webapp does recompile, but, the changes I made to node_modules/package1/index.js are NOT being picked up by the recompilation triggered by /webpack-dev-server/invalidate. And that's a problem for me.

Yep, understand, watchOptions.ignore modify built-in webpack filesystem and says webpack ignore any changes (exclude initial), so even manually changes do not work

Why you need this, what is the real use case, because I find it some weird

tiagobento commented 3 years ago

@alexander-akait I understand, thanks for the clarification.

Yes, indeed my use case is not normal, so I'll try to explain it in detail. Perhaps other people are facing this issue too, given that monorepos and TypeScript are becoming more popular everyday.

I am one of the maintainers of Kogito Tooling. This repo is a monorepo using Lerna, Webpack and TypeScript as the standard technology stack. One of the packages on this monorepo is called online-editor, and it's a standard React-based web application. However, some of its dependencies are, in fact, other packages on this same monorepo.

Here's a list of all packages on which online-editor depends:

As you can imagine, while developing the online-editor webapp, we often need to modify one of its dependency packages. However, since they're all written in TypeScript, after making a change we need to manually rebuild these packages (using webpack) to generate their new JS bundle.

What I'm trying to do is: I'm trying to make the dependency packages of online-editor be automatically rebuilt when something changes in any of them. To do that, I'm using chokidar to watch their directories and, once a change is detected, I trigger a command to rebuilt the changed package along with the packages that depend on it.

When a change happens on a dependency package, we actually need to rebuild not only itself, but also its dependents. And this can take a while to run. Also, many files get deleted/changed/recreated in the process, and that confuses webpack-dev-server watchers. In fact, during the rebuild, webpack-dev-server keeps trying to recompile like crazy, since it's detecting many changes on the files it's watching.

These many recompilations happening very quickly, often lead to the last attempt to fail, since it started before all the dependency packages were done rebuilding, and the online-editor webapp ends up broken. The only way to solve that is to force an invalidation using the invalidation route. However, that's not ideal, since many unnecessary computation is happening by trying to recompile the online-editor webapp without all its dependency packages being rebuilt. And the development workflow gets slower.

My solution was to try and remove the watching capabilities of webpack-dev-server by using watchOptions.ignored. And it worked! When a change happens on one of the dependency packages of online-editor, the watchers I created using a custom script actually recognise the change and rebuild the necessary packages. My problem now is trying to manually recompile the online-editor webapp. And that's why I wanted to use the invalidation route.

I want my dev workflow to be like this:

  1. Launch online-editor webapp.
  2. Make a change on one of its dependency packages.
  3. Automatically rebuild this dependency package and its dependents. (done by my custom script)
  4. Have webpack-dev-server NOT recompile while these packages are rebuilding.
  5. Wait until all the necessary packages are properly rebuilt.
  6. Recompile the online-editor webapp with the rebuilt version of its dependency packages.

Phew! 🥵

Sorry for the long text.

Hope that gives you an idea of the problem I'm trying to solve. It can be that another solution already exists and I'm not aware of it. But with the knowledge I have, what I'm proposing seems to be a good way to solve it :)

Please let me know what you think, and thanks in advance!

alexander-akait commented 3 years ago

Why do not use incremental builds for typescript? Only changed files will be emitted by typescript and webpack rebuild only necessary modules, also webpack@5 have built-in memory/filsystem cache, so it will be very fast

In fact, during the rebuild, webpack-dev-server keeps trying to recompile like crazy, since it's detecting many changes on the files it's watching.

You can use poll https://github.com/webpack/webpack/blob/master/schemas/WebpackOptions.json#L3645 (with delay)

tiagobento commented 3 years ago

@alexander-akait We generate our bundle files using ts-loader, since we want to keep checking for type errors during compilation. I'm not sure that would apply to such case. Also, my online-editor webapp doesn't have a direct reference to the source code of its dependency packages. It only references their bundled file on node_modules/a-dependency-package/dist/index.js, for instance. If you have additional resources you can provide me, that would be great.

About poll, I tried increasing the poll limit to a really big number, but then I suffer from the same problem of watchOptions.ignored = "**/*.*".

Anyway.. for the moment, I'll just accept the unnecessary computation that is going to happen with no watchOptions.ignored or poll configurations. But, to be certain that my app won't end up in a broken state, I will trigger a manual invalidation when my dependency packages finish rebuilding.

I really want to thank you for your availability on replying to these messages! Hopefully this will serve for reference for other people too! 😄

alexander-akait commented 3 years ago

What is webpack version? You can add this file as buildDependecies and webpack will reload application, watchOptions.ignored = "**/*.*" is hack, webpack and webpack-dev-server were not designed for this use case

alexander-akait commented 3 years ago

@tiagobento friendly ping

tiagobento commented 3 years ago

Hi @alexander-akait! Thanks for the ping and sorry for not getting back to your earlier.

It's been a while I don't try solving this issue again. Unfortunately I didn't have time to try your suggestion and the project changed quite a bit. We're planning to migrate from Webpack 4 to Webpack 5 soon, so I'll probably have more to add once we're finished with the upgrade.

I appreciate your help and feel free to close this issue if needed. I can open a new one or reopen this one when the time comes!

alexander-akait commented 3 years ago

Yep, feel free to open an issue if you faced with the problem, thanks for answer