Files created right before watching starts make watching go into a loop

[KidkArolis]

I've noticed this strange behaviour where watcher loops for 10 seconds due to this code:

if(data) {
  var ts = data[0] === data[1] ? data[0] + FS_ACCURENCY : data[0];
  if(ts > startTime)
    watcher.emit("change", data[1] + FS_ACCURENCY);
}

https://github.com/webpack/watchpack/blob/7efdd93aff0c991a766af886ccc16324401a202f/lib/DirectoryWatcher.js#L202-L205

What I'm doing in my script is basically creating a file in a temporary directory /temp/entry.js, creating a webpack compiler and run compiler.watch(). Now for 10 seconds (which is the default value of FS_ACCURENCY) this is what's happening:

• compilation succeeds
• Watching.prototype._done gets called
• recalling Watching.prototype.watch
• which immediately triggers a change event for the /temp/entry.js file, because it's < 10 seconds old
• back to step 1

Any pointers on how to fix this would be helpful. For example, this could be fixed by modifying the above code to this, but I'd need to stare at this for longer to understand what this block of code is conceptually responsible for.

if(ts > startTime + FS_ACCURENCY) {

[sokra]

FS_ACCURENCY should automatically adjust to your file system accuracy. This may not happen fast enough if you have few files and the files are created unlucky on a timestamp modulo 10s.

The watching may loop in a unlucky case, but this should not result in a different compilation hash. I. e. the webpack-dev-server doesn't trigger a update if the hash is equal.

[KidkArolis]

Ah, but Im experiencing this issue 100% of the time, because I am programatically creating the entry.js file and starting the watch process on it right away. So for 10s it keeps rebuilding continuously. On Fri, 6 May 2016 at 13:12, Tobias Koppers notifications@github.com wrote:

FS_ACCURENCY should automatically adjust https://github.com/webpack/watchpack/blob/master/lib/DirectoryWatcher.js#L230-L239 to your file system accuracy. This may not happen fast enough if you have few files and the files are created unlucky on a timestamp modulo 10s.

The watching may loop in a unlucky case, but this should not result in a different compilation hash. I. e. the webpack-dev-server doesn't trigger a update if the hash is equal.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/webpack/watchpack/issues/25#issuecomment-217424240

[KidkArolis]

Would be nice to resolve this without having to fork webpack or delay watching for 10s. I wonder if I could create a file with an older timestamp.. On Fri, 6 May 2016 at 18:29, Karolis Narkevičius karolis.n@gmail.com wrote:

Ah, but Im experiencing this issue 100% of the time, because I am programatically creating the entry.js file and starting the watch process on it right away. So for 10s it keeps rebuilding continuously. On Fri, 6 May 2016 at 13:12, Tobias Koppers notifications@github.com wrote:

FS_ACCURENCY should automatically adjust https://github.com/webpack/watchpack/blob/master/lib/DirectoryWatcher.js#L230-L239 to your file system accuracy. This may not happen fast enough if you have few files and the files are created unlucky on a timestamp modulo 10s.

The watching may loop in a unlucky case, but this should not result in a different compilation hash. I. e. the webpack-dev-server doesn't trigger a update if the hash is equal.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/webpack/watchpack/issues/25#issuecomment-217424240

[Jessidhia]

@sokra "FS_ACCURENCY" only adjusts itself when you get an actual onChange event. When you first fire compiler.watch(), and the compilation is fast enough (say, one file with no bundled dependencies), assuming you saved the file in the last few seconds, watchpack will keep emitting change events until the difference gets large enough.

When chokidar is using fswatch (OSX), FS_ACCURENCY eventually settles on 2000 if you trigger some real change events (not sure why, fsevent timestamps are multiples of 1000).

Maybe FS_ACCURENCY should get a first adjustment on the nextTick handler; at least that would reduce the worst case to 2s as opposed to 10s. If the build is really fast (say, ~400ms per build), you can easily get 20+ rebuilds at the start if you touched any file before starting the watch process.

It'd also be nice if it got renamed to FS_PRECISION, but that's going off-topic :)

[rochdev]

We are having the same problem. A workaround we found is to manually change the mtime of the file before it gets picked up by Webpack, but it feels like just adding a hack on top of a hack.

[hrimhari]

This problem has been haunting our team for a while. Files are being stored on doInitialScan() 10 seconds in the future and this generates multiple webpack-dev-middleware builds until the future becomes the past. It seems to me that watchpack should compare mtimes between nextTick()s instead of trying to guess if the mtime is in the future or in the past.

[sokra]

Watchpacks strategy is: if not sure if it's a change, thread it as change. With low fs accuracy files could have changed even if mime is equal.

Normally multiple rebuilds shouldn't cause issues and this cause should be cached by the caller by comparing hash. I.e. the webpack CLI doesn't display stats output for equal hash.

Nevertheless this may could be fixed in watch pack by storing the time of the initial scan/watcher installation and useing this time.

[hrimhari]

How is FS_ACCURENCY helping in that case? If the filesystem doesn't provide the precision necessary to detect the change, FS_ACCURENCY will only add X milliseconds to that lack of precision. If mtime didn't change because of lack of precision, adding FS_ACCURENCY to it won't make it any different. The only reason I can imagine is when trying to guess if the mtime is in the future or not, and what I propose is to stop doing that and just act upon the change in mtime itself.

[hrimhari]

FYI, here's an extract of what we see if we have a recent change before starting webpack-dev-middleware. I put a loop printing the current date in the background and added some console.logs for debugging purposes showing arguments for the compiler plugins. Note that the hash remains the same and the multiple builds happening until startTime is greater than mtime + FS_ACCURENCY:

Tue 18 Oct 2016 11:33:20 EDT
webpack built a6a4935a84d4fc7fddd3 in 3823ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 3823ms
                               Asset     Size  Chunks             Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB          [emitted]
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB          [emitted]
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB          [emitted]
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0  [emitted]
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1  [emitted]
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2  [emitted]  app
                         favicon.ico  7.41 kB          [emitted]
                       vendor.dll.js  3.92 MB          [emitted]
                          index.html  6.61 kB          [emitted]
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks             Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB          [emitted]
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
Tue 18 Oct 2016 11:33:21 EDT
invalid
{ '0': '/Users/fcarasso/Dev/console-proto/src/main.js',
  '1': 1476804803000 }
webpack: bundle is now INVALID.
invalidAsync
{ '0':
   Watching {
     startTime: 1476804802113,
     invalid: false,
     error: null,
     stats:
      Stats {
        compilation: [Object],
        hash: 'a6a4935a84d4fc7fddd3',
        startTime: 1476804797495,
        endTime: 1476804801318 },
     handler: [Function],
     watchOptions: { aggregateTimeout: 200 },
     compiler:
      Compiler {
        _plugins: [Object],
        outputPath: '/Users/fcarasso/Dev/console-proto/dist',
        outputFileSystem: [Object],
        inputFileSystem: [Object],
        recordsInputPath: undefined,
        recordsOutputPath: undefined,
        records: [Object],
        fileTimestamps: [Object],
        contextTimestamps: [Object],
        resolvers: [Object],
        parser: [Object],
        options: [Object],
        context: '/Users/fcarasso/Dev/console-proto',
        name: 'client',
        watchFileSystem: [Object],
        _lastCompilationFileDependencies: [Object],
        _lastCompilationContextDependencies: [] },
     running: true,
     watcher: null },
  '1': [Function: next] }
invalid
{}
webpack building...
webpack built a6a4935a84d4fc7fddd3 in 227ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 227ms
                               Asset     Size  Chunks       Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2       app
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks       Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
invalid
{ '0': '/Users/fcarasso/Dev/console-proto/src/main.js',
  '1': 1476804803000 }
webpack: bundle is now INVALID.
invalidAsync
{ '0':
   Watching {
     startTime: 1476804802571,
     invalid: false,
     error: null,
     stats:
      Stats {
        compilation: [Object],
        hash: 'a6a4935a84d4fc7fddd3',
        startTime: 1476804802113,
        endTime: 1476804802340 },
     handler: [Function],
     watchOptions: { aggregateTimeout: 200 },
     compiler:
      Compiler {
        _plugins: [Object],
        outputPath: '/Users/fcarasso/Dev/console-proto/dist',
        outputFileSystem: [Object],
        inputFileSystem: [Object],
        recordsInputPath: undefined,
        recordsOutputPath: undefined,
        records: [Object],
        fileTimestamps: [Object],
        contextTimestamps: [Object],
        resolvers: [Object],
        parser: [Object],
        options: [Object],
        context: '/Users/fcarasso/Dev/console-proto',
        name: 'client',
        watchFileSystem: [Object],
        _lastCompilationFileDependencies: [Object],
        _lastCompilationContextDependencies: [] },
     running: true,
     watcher: null },
  '1': [Function: next] }
invalid
{}
webpack building...
Tue 18 Oct 2016 11:33:22 EDT
webpack built a6a4935a84d4fc7fddd3 in 220ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 220ms
                               Asset     Size  Chunks       Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2       app
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks       Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
invalid
{ '0': '/Users/fcarasso/Dev/console-proto/src/main.js',
  '1': 1476804803000 }
webpack: bundle is now INVALID.
invalidAsync
{ '0':
   Watching {
     startTime: 1476804803016,
     invalid: false,
     error: null,
     stats:
      Stats {
        compilation: [Object],
        hash: 'a6a4935a84d4fc7fddd3',
        startTime: 1476804802571,
        endTime: 1476804802791 },
     handler: [Function],
     watchOptions: { aggregateTimeout: 200 },
     compiler:
      Compiler {
        _plugins: [Object],
        outputPath: '/Users/fcarasso/Dev/console-proto/dist',
        outputFileSystem: [Object],
        inputFileSystem: [Object],
        recordsInputPath: undefined,
        recordsOutputPath: undefined,
        records: [Object],
        fileTimestamps: [Object],
        contextTimestamps: [Object],
        resolvers: [Object],
        parser: [Object],
        options: [Object],
        context: '/Users/fcarasso/Dev/console-proto',
        name: 'client',
        watchFileSystem: [Object],
        _lastCompilationFileDependencies: [Object],
        _lastCompilationContextDependencies: [] },
     running: true,
     watcher: null },
  '1': [Function: next] }
invalid
{}
webpack building...
webpack built a6a4935a84d4fc7fddd3 in 207ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 207ms
                               Asset     Size  Chunks       Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2       app
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks       Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
Tue 18 Oct 2016 11:33:23 EDT
Tue 18 Oct 2016 11:33:24 EDT
Tue 18 Oct 2016 11:33:25 EDT
^C

[mischkl]

We are experiencing the endless rebuilding problem even for files that weren't created shortly before watching started. This is a problem that has affected us with Webpack 1 as well as 2, and occurs sporadically but often enough that it makes working with webpack-dev-server a chore and it almost makes one long for the days of grunt-watch and BrowserSync. ;) Here is a link to our config: https://gist.github.com/mischkl/cf7ab4b10df9763aa698a5b885f700cd

This is on Windows 7 Enterprise x64 with Node 6.9.1, and seems to primarily happen when IntelliJ IDEA is open.

I should also note that we have seen the problem when using angular-cli with the default settings as well. The fact that it only seems to affect a subset of users makes me strongly suspect that this is a Windows/Intellij/WebStorm-related issue.

[fenomas]

For anyone else tearing out hair over this, backdating the relevant files by ten seconds is an awful hack but it worked for me:

var f = path.resolve('someFile')
var now = Date.now() / 1000
var then = now - 10
fs.utimes(f, then, then, function (err) { if (err) throw err })

Update - This fix stopped working for me at some point, but Yassky's solution further down the page works great.

[atinux]

You just made my day @andyhall

I confirm that his solution works like a charm!

[trsh]

Same here, check https://github.com/webpack/watchpack/issues/53. My explanation why this happens is very detailed.

[AlexGalays]

Webpack also has this bug if the files are created early in the process but AFTER calling webpack(options).watch().

Strangely enough, doing a command line webpack --watch (with the exact same options) doesn't display the same issue.

[trsh]

@sokra from your last comment, what is the conclusion? Address this to webpack and Close issue, or the fix your are talking about is on the way / planed. Right now it's moving nowhere, very popular and annoying :). I would gladly make a P.R., but I did not understand your strategy.

[trsh]

@AlexGalays for me, I just need to edit the some script, wait for at last 10 secs, and then run webpack --watch.

[ericclemmons]

I can confirm that https://github.com/webpack/watchpack/issues/25#issuecomment-287789288 fixes the problem, but would need help figuring out why the webpack code has this problem.

The best I can figure is this?

https://github.com/webpack/webpack/blob/c8732c8d157ed56542d521a557dda7ff7ad12ffb/lib/Compiler.js#L44

[hrimhari]

As I mentioned before, it seems to me that FS_ACCURACY doesn't really help anything and creates all this hassle. The strategy I'd suggest would be to remove it.

[yessky]

This issue also confused me for a while. finally I found a trick to fix it.

  const compiler = webpack(webpackConfig);
  const timefix = 11000;
  compiler.plugin('watch-run', (watching, callback) => {
    watching.startTime += timefix;
    callback()
  });
  compiler.plugin('done', (stats) => {
    stats.startTime -= timefix
  })

also a npm package is available https://github.com/yessky/webpack-mild-compile, it works with webpack v2/3/4.

[Robinfr]

This is rather problematic when working with Lerna where you link several projects together sometimes.

[gmcdev]

yessky's hack is the jam:

• On compiler watch-run, offset the start time by 11s
• On compiler done, restore the start-time

The same fix is here, as a plugin.

[mcpherson-sa]

I am experiencing the issue on Ubuntu, have tried tried the mild compile plugin but it doesn't seem to help. Am interested in the fix mention by Andy Hall but am unsure were to apply the code.

[mcpherson-sa]

I added some logging to DirectoryWatcher.js and for my case I don't think the problem is in this code

if(data) {
  var ts = data[0] === data[1] ? data[0] + FS_ACCURENCY : data[0];
  if(ts > startTime)
    watcher.emit("change", data[1] + FS_ACCURENCY);
}

emit doesn't get called here when I change my file. I put logging code in all the other places where emit is called an seems to only be called once. So I am thinking the problem is elsewhere. I seem to be getting only one compile but multiple browser reloads.

[hrimhari]

@mcpherson-sa what's your understanding after reading the previous comments ?

[mcpherson-sa]

@hrimhari - I think I might be seeing a problem which is different from what some other people are seeing, as I am not seeing multiple compiles just multiple reloads. What behavior are you observing?

[felipenmoura]

I'm seeing this problem like this:

// fs is fs-extra
fs.ensureFileSync('./src/client/main/index.js'))

compiler.watch(SETTINGS, function () { /* ... */} )

If I put the compiler inside a setTimeout of 10 secs, it works. If the delay is 9 secs, it goes into a loop 100% of times. That only happens if the file was created. If the file already existed, it works normally :/

[felipenmoura]

I managed to fix it using the tip I found here: https://github.com/ckeditor/ckeditor5-dev/issues/346

const now = Date.now() / 1000;
const then = now - 11;
// set the changing time to 11 seconds ago
fs.utimesSync(path, then, then)

[yessky]

@felipenmoura try better workaround

[fenomas]

Chiming in to add that the solution I posted further up the thread no longer works for me, but @yessky 's works great.

The fix can be added directly into a webpack.config.js file like this:

function TimeFixPlugin() {
    this.apply = function (compiler) {
        var timefix = 11000
        compiler.plugin('watch-run', (watching, callback) => {
            watching.startTime += timefix
            callback()
        })
        compiler.plugin('done', (stats) => {
            stats.startTime -= timefix
        })
    }
}

config.plugins.push(new TimeFixPlugin())

[pi0]

For anyone having this problem, you can use time-fix-plugin by @egoist which is fully compatible with webpack 4. (Current method does not works anymore with >= 4.0.0)

[alexander-akait]

Some body can send PR with tests? We open to fix this, just ping me :+1:

[mixtur]

Why not just call ensureFsAccuracy with some (current?) file's mtime upfront? It will effectively turn this problem to heisenbug)

[yessky]

webpack-mild-compile now supports webpack v2/3/4. I really hope watchpack team solve this issue, so we can remove this workaround

[pi0]

@yessky Sorry but I think webpack-mild-compile needs little more fixes for wp4. (You can try debugging. watching is probably undefined for onWatchRun hook). Please take a look at this hack)

[yessky]

@pi0 Thanks. watchRun callback receive a compiler instance instead of 'watching' under v4.

[HIRANO-Satoshi]

I'm using wp3.

The webpack-mild-compile and time-fix-plugin won't work. But andyhal's workaround works.

[yessky]

@HIRANO-Satoshi try again, webpack-mild-compile should works now

[HIRANO-Satoshi]

@yessky I comfirmed your webpack-mild-compile worked for wp3. Thanks.

[EECOLOR]

I don't understand the issue completely, but this issue seems to be open for quite some time. Will there be a fix?

[nigelkong]

If time-fix-plugin still does not solve your problem, adding this together with the time-fix-plugin solved the issue for me:

plugins: [ new webpack.WatchIgnorePlugin([ /\.js$/, /\.d\.ts$/ ]) ], Thanks to @ORESoftware at this issue!

[shcallaway]

I started experiencing the infinite compilation loop after changing the macOS date/time preferences in order to reproduce a date-related bug in my app. Just a warning to anyone else who might run into the same issue!

[aseem2625]

Working solution for me: Compare with last stored hash.

const curHash = stats.hash;

if (lastHash !== curHash) {
  lastHash = curHash;

  return cb && cb();
}

[pi0]

@aseem2625 I think with your solution, we would have 2 compiles instead of a loop. (Second build->hash is extra)

[aseem2625]

@pi0 So, in my cb() I'm doing stuff with the file system(not editing the filed watched by webpack though). This apparently is the reason which is causing it to run infinite times (Because if I put empty fn. for cb, it just runs once). So, only possible way was to check with last stored hash. Ya, it does run 2 times, which I'm not sure why, but there are no side effects as such.

[yuchonghua]

Still quite looking forward to this problem has a solution as soon as possible

[yuchonghua]

Looking forward to the master to fix this problem, contribute a pr to webpack/watchpack

[HadiChen]

Hope that there can be a solution

[yuchonghua]

Watchpacks strategy is: if not sure if it's a change, thread it as change. With low fs accuracy files could have changed even if mime is equal.

Normally multiple rebuilds shouldn't cause issues and this cause should be cached by the caller by comparing hash. I.e. the webpack CLI doesn't display stats output for equal hash.

Nevertheless this may could be fixed in watch pack by storing the time of the initial scan/watcher installation and useing this time.

I am looking forward to the official solution to this problem.

[sokra]

old issue

problem is kind of expected behavior

we reduced the startup filesystem accuracy to 1 second which should reduce the problem

[hrimhari]

Given the number of people complaining, "expected behavior" seems inappropriate as a classification.

Why not just get rid of FS_ACCURACY?