--watch does not work on mounted drive on Linux

[travelmassive]

I'm using esbuild to compile JS for my Phoenix Elixir app.

My development environment is:

• A Mac (OSX 13.4) for VSCode and Web Browser
• Linux (Ubuntu 20.04) for Elixir, my database, esbuild, etc.
• Linux runs on my Mac as Virtual Machine via Parallels Virtual Machine.
• My ~/Documents/Projects folder is mounted to my Linux /mnt/projects folder using Parallels shared folders

Problem:

On my Linux environment, esbuild --watch goes into a loop of (falsely) detecting files that have changed and rebuilding them, until some kind of file handler limit is exhausted and esbuild stops + my Elixir code reloader crashes.

The same command works when run on my Mac in the same folder.

esbuild command:

export NODE_PATH="`pwd`/deps:`pwd`/assets/vendor"
esbuild assets/js/app.js --bundle --target=es2017 --outdir=./priv/static/assets --external:"/fonts/*" --external:"/images/*"  --sourcemap --minify --watch

On my Mac (OSX 13.4, esbuild 0.19.3), esbuild works as expected (I change one file, example.js and it rebuilds):

ian@Ians-MacBook-Pro-2 myapp % export NODE_PATH="`pwd`/deps:`pwd`/assets/vendor"
ian@Ians-MacBook-Pro-2 myapp % esbuild assets/js/app.js --bundle --target=es2017 --outdir=./priv/static/assets --external:"/fonts/*" --external:"/images/*"  --sourcemap --minify --watch
[watch] build finished, watching for changes...
[watch] build started (change: "assets/js/hooks/example.js")
[watch] build finished

On Linux (Ubuntu 20.04, esbuild 0.19.3), without changing any code, esbuild loops through many different files (that haven't changed) before eventually being "unable to resolve ./assets/js/app.js", then other apps reading the folder report errors.

ian@ubuntu:/media/psf/Projects/myapp# export NODE_PATH="`pwd`/deps:`pwd`/assets/vendor"
ian@ubuntu:/media/psf/Projects/myapp# esbuild assets/js/app.js --bundle --target=es2017 --outdir=./priv/static/assets --external:"/fonts/*" --external:"/images/*"  --sourcemap --minify --watch
[watch] build finished, watching for changes...
[watch] build started (change: "deps/phoenix_html/priv/static/phoenix_html.js")
[watch] build finished
[watch] build started (change: "assets/js/hooks/example.js")
[watch] build finished
[watch] build started (change: "assets/js/hooks/example.js")
[watch] build finished
[watch] build started (change: "deps/phoenix_live_view/priv/static/phoenix_live_view.esm.js.map")
[watch] build finished
[watch] build started (change: "assets/js/hooks/logout.js")
[watch] build finished
[watch] build started (change: "assets/js/hooks/rsvp.js")
[watch] build finished
[watch] build started (change: "assets/js/hooks/example.js")
✘ [ERROR] Could not resolve "./assets/js/app.js"

1 error
[watch] build finished

At this point, other things start breaking on my VM.

Shell output (from direnv)

direnv: error LoadConfig() Getwd failed: "getwd: no such file or directory"

Output from Phoenix Elixir App (0.19.5) also dies...

** (File.Error) could not get current working directory nil: no such file or directory
    (elixir 1.15.5) lib/file.ex:1567: File.cwd!/0
    (elixir 1.15.5) lib/path.ex:166: Path.expand/1
    (mix 1.15.5) lib/mix/project.ex:752: Mix.Project.app_path/1
    (mix 1.15.5) lib/mix/project.ex:810: Mix.Project.consolidation_path/1
    (phoenix 1.7.7) lib/phoenix/code_reloader/server.ex:180: Phoenix.CodeReloader.Server.mix_compile/5
    (phoenix 1.7.7) lib/phoenix/code_reloader/server.ex:74: anonymous fn/4 in Phoenix.CodeReloader.Server.handle_call/3
    (phoenix 1.7.7) lib/phoenix/code_reloader/server.ex:295: Phoenix.CodeReloader.Server.proxy_io/1
    (phoenix 1.7.7) lib/phoenix/code_reloader/server.ex:72: Phoenix.CodeReloader.Server.handle_call/3
    (stdlib 5.0.2) gen_server.erl:1113: :gen_server.try_handle_call/4
    (stdlib 5.0.2) gen_server.erl:1142: :gen_server.handle_msg/6
    (stdlib 5.0.2) proc_lib.erl:241: :proc_lib.init_p_do_apply/3

If I run the esbuild command on Linux without --watch, the build runs fine without issues.

I would expect it shouldn't matter that I'm running esbuild on a mounted file system?

Thanks for any help — Ian

[evanw]

This is probably something that you are going to have to debug on your end, perhaps by building esbuild from source with additional debugging code added. The file watcher is polling-based and tries to use stat syscalls for performance when detecting changes. Specifically various properties returned by stat are merged into something esbuild calls a "mod key" that represents the file's state at rest. On Unix-like operating systems, esbuild's mod key combines the following information (see the code for details):

• The inode
• The file size in bytes
• The modification timestamp
• The permissions mode
• The user id of the file

When any of those properties change in between stat calls, esbuild's watch mode will trigger another build. There are also conditions when esbuild doesn't use mod keys such as when the modification timestamp returned by the operating system is zero or when the modification timestamp is too new (since modification timestamps for some file systems have a large granularity).

It's possible that the virtual file system you're using is mutating one of those properties on every stat call which would then cause esbuild to think that the file system is always changing. It's also possible that there's something else going on instead. This is just a guess on my end after reading what you wrote.

[travelmassive]

Thanks for the tips.

I manually built esbuild from source and printed the return values of ModKey (inode, mtime_sec, mtime_nsec, etc) to take a look. There's nothing interesting here — the values are same for each file between polls (without modifying a file).

I also made a copy of my app on my native filesystem on Linux, and esbuild works as expected here.

Here's some example output, debugging the values in modkey_unix.go:modKey():

Mac:

path:                   /Users/ian/Documents/Projects/myapp/assets/js/hooks/example.js
stat.Ino:               130632349
stat.Size:              532
int64(stat.Mtim.Sec):   1696025390
int64(stat.Mtim.Nsec):  614540869
uint32(stat.Mode):      33188
stat.Uid:               501

Linux (via mounted folder):

path:                   /media/psf/Projects/myapp/assets/js/hooks/example.js
stat.Ino:               2615497
stat.Size:              532
int64(stat.Mtim.Sec):   1696025390
int64(stat.Mtim.Nsec):  0
uint32(stat.Mode):      33188
stat.Uid:               1000

Linux (from a copy on root (native, non-mounted) filesystem):

path:                   /home/ubuntu/myapp/assets/js/hooks/example.js
stat.Ino:               1214891
stat.Size:              532
int64(stat.Mtim.Sec):   1696736918
int64(stat.Mtim.Nsec):  411436975
uint32(stat.Mode):      33188
stat.Uid:               1000

When I run esbuild on the mounted filesystem in Linux with --log-level=verbose, I noticed that esbuild is reporting it was resolving in the root folder "/". Could this be a clue?

Resolving import "./assets/js/app.js" in directory "/" of type "entry-point"

  Read 26 entries for directory "/"
  Read 26 entries for directory "/"
  Read 26 entries for directory "/"
  Failed to read directory "/assets": open /assets: no such file or directory
  Failed to read directory "/assets"
  Failed to read directory "/assets/js"
  Attempting to load "/assets/js/app.js" as a file
    Failed to read directory "/assets/js": open /assets/js: no such file or directory
  Attempting to load "/assets/js/app.js" as a directory
    Failed to read directory "/assets/js"
    Failed to read directory "/assets/js/app.js"

ReadDirectory /assets/js <-- my own debug Println in fs_real.go:ReadDirectory()
✘ [ERROR] Could not resolve "./assets/js/app.js"