[WSL2] File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps

SteveSandersonMS commented 4 years ago

WSL2 is really close to being a perfect runtime environment for server apps being developed in Windows. Great job! One missing feature however is breaking a core part of the developer flow.

For sources stored on the Windows filesystem, any changes made by Windows applications such as Visual Studio do not trigger any file change notifications as far as Linux apps are concerned. This means that all "live rebuild"-type tools don't work (examples: webpack --watch, jekyll --interactive, and Tilt.dev) when running under WSL2. This unfortunately renders many modern dev workflows unviable.

Notes:

I strongly expect this is already a known issue as hinted here. However I was unable to find any open GitHub issue that specifically tracks this. #1956 is about a different scenario, #4064 isn't specifically about change notifications, #4224 is closed, #4293 doesn't fix it.
I'm aware the WSL team suggests that with WSL2 it's good to store your application sources inside the Linux filesystem instead of the Windows one. However I don't think that's a viable solution since the whole point of WSL (for me) is to be able to use Windows-based editors such as full VS with a Linux dev environment. If it was sufficient to use Linux-based editors, I'd just use Mac/Linux as my actual OS and would have no need for WSL.
This did work great on WSL1
Changing fs.inotify.max_user_watches doesn't affect this, since the issue is about changes originating on the Windows side.

Bug report template

Your Windows build number: 10.0.19033.1
What you're doing and what's happening:

This applies to all tools that listen for file change notifications, but as an example take webpack. Repro steps:
- In your Windows filesystem, create an empty directory (example: c:\repro), and then add these three files to it
- In a WSL2 Ubuntu 18.04 environment, install Node and NPM: sudo apt-get install nodejs npm
- Still in WSL2, go into the directory from earlier: cd /mnt/c/repro
- Restore NPM dependencies: npm i
- Run Webpack in watch mode: npm run build:watch. Wait a few seconds until it completes the first build. It will now be waiting for further changes to your source files.
- In a Windows application (e.g., Notepad or Visual Studio), open c:\repro\index.js and save some change to it. For example, change 'Hello, world' to 'Hello, world 2'.
What's wrong / what should be happening instead:

Expected behavior: Webpack should see the change and rebuild. That is, you'll see it log information about another build, and the output in dist/bundle.js will be updated.

Actual behavior: Webpack doesn't respond at all, because there's no file change notification.

Finally I understand that the fix for this is likely to be "add file watch capabilities to the Plan9 server", and you may feel this is already being tracked by #4064. However #4064 describes a more obscure symptom of this missing feature and makes it sound like an intermittent issue. What I'm reporting here is not intermittent at all, and is a pretty mainstream scenario (using tools like webpack --watch). Thanks!

adamkerz commented 1 year ago

Is this ever going to get fixed? If not, you should at least close it as wontfix so that people can move on with their lives (probably to Linux).

helgatheviking commented 1 year ago

Lol . I did move on to pure Ubuntu

jankap commented 1 year ago

We started to use the WSL filesystem as recommended. Clone into \\wsl$\ubuntu. From windows and open either that path in Windows tools or work inside WSL, or even better in devcontainers. It's pretty fast.

raffaeler commented 1 year ago

@jankap the problem here is not the speed but getting the file system changes notification from an app runing under linux.

jankap commented 1 year ago

@jankap the problem here is not the speed but getting the file system changes notification from an app runing under linux.

You get the notifications in Linux that way. The other way round could be problematic. I think the question is where the notifications and native speeds have to be available. Windows? Use window FS. Linux or docker based devcontainers? Use Linux FS which can initially be accessed via the network share to clone the code etc.

Nevertheless, this issue should be fixed. I'm not saying there is none.

dig12345 commented 1 year ago

I tried to work around this by setting up a windows file share and mounting with cifs-utils. I was able to mount it however there seems to be some strange bug with running windows executables from the wsl command line within a folder on the mount. If I run ping.exe outside of the mount subfolder it works. Inside a subfolder within the mount I get the following cryptic output: /mnt/c/windows/system32/ping.exe: Invalid argument . Here is the mount command I am using mount -t cifs -o username=linux_user,password='pass',vers=3.0 //192.168.0.100/code_linux /mnt/smb/code. The only clue I have is that the issue only happens when you are at least 1 folder deep inside the mount. At the root of the mount, windows executables work fine.

If I can resolve this execution issue, I think SMB would be a decent workaround until inotify gets fixed....

PS The reason /mnt/c fix is important, is that storing files on linux is risky since WSL distros have a tendency to spontaneously lose all user files. It also means files can persist across distro changes etc..

Fleker commented 1 year ago

I just ran into this problem this evening when spinning up a new Angular project after updating my default to WSL2. Thankfully I still have the original and jumped back to run the project with live reload.

potchy commented 1 year ago

We started to use the WSL filesystem as recommended. Clone into \\wsl$\ubuntu. From windows and open either that path in Windows tools or work inside WSL, or even better in devcontainers. It's pretty fast.

I ended up doing this as well. It's kinda awkward at first, but it's the path of least resistance and it works great once you get used to it. If you don't want to setup .gitconfig again, just include it in /home/{your Linux user here}/.gitconfig like so.

[include]
    path = /mnt/c/Users/{your Windows user here}/.gitconfig

jrhager84 commented 1 year ago

Dang - Still no movement? Unfortunate.

jonstrutz11 commented 1 year ago

I came here because VS Code would not automatically refresh source control upon file changes since I have all my project files stored in the windows filesystem - specifically, so I can have them backed up via OneDrive.

I found a solution where you can store your project files on the Linux filesystem and then regularly sync them to your windows filesystem OneDrive folder so that they get backed up. See this blog post from Stephen Rees-Carter for details on how to set it up.

(fyi I deleted the mysql database backup part of the script he provides - I just used it to sync files to a folder in my OneDrive and used Task Scheduler to have it run daily)

Not perfect and takes an hour or two to set up the first time, but after that you don't need to worry about it and it seems to work quite well so far.

kiweezi commented 1 year ago

It's annoying that this still isn't fixed and the fact there is no communication from the team is pretty bad.

What seemed to work for me was creating a devcontainer and then using the Clone Respository in Container Volume... function in vscode with the devcontainers extension. Makes git run much faster, plus you don't have to worry about setting up your environment in WSL etc.

Devcontainers are relatively easy to setup, you just need WSL and Docker installed. The vscode extension for devcontainers can help you create a devcontainer configuration file really quickly.

Not sure if this has been said before, but hopefully it helps someone.

sdserage commented 1 year ago

Is this ever going to get fixed? If not, you should at least close it as wontfix so that people can move on with their lives (probably to Linux).

If only that was an option, my company is working on Windows exclusively and my coworkers are more likely to pull their own teeth out before switching off of Windows.

Is there really no progress for this issue still?

PedramMarandi commented 1 year ago

This has been open since 2019 with no official response.

PaulOst commented 1 year ago

"[WSL2] File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps #4739"

Is there a fix for this anywhere in our future? If you're never going to fix this, then say so!

ManfredLange commented 1 year ago

Perhaps, not fixing it is a subtle message from Microsoft that we all should move to Linux proper ... ? If we did, this problem would certainly be resolved. 🤔

mindplay-dk commented 1 year ago

Can someone lock this thread, please? Getting tired of "me too" notifications, and it's fairly clear the maintainers aren't going to respond, so you might as well block this noise and just let us know if it ever gets fixed.

Or if you're not going to address this, maybe just close the thread.

Everyone knows what the problem is, and only MS can solve it - no reason to waste anybody else's time then.

dig12345 commented 1 year ago

Can someone lock this thread, please? Getting tired of "me too" notifications, and it's fairly clear the maintainers aren't going to respond, so you might as well block this noise and just let us know if it ever gets fixed.

Or if you're not going to address this, maybe just close the thread.

Everyone knows what the problem is, and only MS can solve it - no reason to waste anybody else's time then.

Umm click unsubscribe? Me too comments demonstrate how widespread the issue is and allow for discussion around workarounds/root cause. If you're not interested, simply unsubscribe.

jankap commented 1 year ago

This has been open since 2019 with no official response.

I'm curious, did anybody ever addressed this to MS engineers directly? Are they aware of that?

jankap commented 1 year ago

Another comment: Notifications work when using volume mounts, not bind mounts. This can be very helpful for build folders:

https://code.visualstudio.com/remote/advancedcontainers/improve-performance#_use-a-targeted-named-volume

raffaeler commented 1 year ago

Another comment: Notifications work when using volume mounts, not bind mounts. This can be very helpful for build folders:

https://code.visualstudio.com/remote/advancedcontainers/improve-performance#_use-a-targeted-named-volume

My main scenario is when you want to keep in sync windows and linux folders with rsync. Did you test this scenario?

omt66 commented 11 months ago

Can someone lock this thread, please? Getting tired of "me too" notifications, and it's fairly clear the maintainers aren't going to respond, so you might as well block this noise and just let us know if it ever gets fixed. Or if you're not going to address this, maybe just close the thread. Everyone knows what the problem is, and only MS can solve it - no reason to waste anybody else's time then.

Umm click unsubscribe? Me too comments demonstrate how widespread the issue is and allow for discussion around workarounds/root cause. If you're not interested, simply unsubscribe.

Ditto. This is a big problem, if anyone doesn't want to hear, they should unsubscribe!

adam-ah commented 7 months ago

Geez, 4 years, still no solution? This bug breaks so many things. How do we escalate it to MS to have a look?

PaulOst commented 7 months ago

Is MS monitoring these issues?

erentar commented 6 months ago

I also desperately need this feature

mattbell87 commented 6 months ago

I also desperately need this feature

Can you run your project inside WSL? That fixes issues like this for me. If you need access from Windows just use \\wsl$ in explorer/files.

raffaeler commented 6 months ago

Can you run your project inside WSL? That fixes issues like this for me. If you need access from Windows just use \\wsl$ in explorer/files.

accessing is not enough, the need is to use rsync to keep the files updated (aka file system notification).

Anyway, on my side I don't care anymore as I moved my dev machine to Ubuntu and resolved this issue radically. The best surprise from this move, was discovering the file system operations on Ubuntu are hugely faster during C++/.NET compile time or nodjs development.

EmmanuelTheCoder commented 6 months ago

I also desperately need this feature

Can you run your project inside WSL? That fixes issues like this for me. If you need access from Windows just use \\wsl$ in explorer/files.

That works but it defeats interoperability. Didn't they say that WSL is interoperable with Windows?

MiikaH commented 6 months ago

That works but it defeats interoperability. Didn't they say that WSL is interoperable with Windows?

Not sure what you mean by that.

You keep your projects in WSL file system, access them via \\wsl.localhost\ . Explorer access, editor & IDE access, docker and everything should pretty much work the same way as if you stored your project on C:\, D:\ etc, but now the files are on Linux ext4 filesystem so performance should be way higher than NTFS and file system notifications work as expected, as stated above.

And if you want to take things one step further you can (on top of WSL) setup your project to use VSCode "Dev Containers" in order to run the project & VSCode in virtual Linux distro of your choosing. Which not only makes it easier to run Linux primary projects on Windows as you can just run the linux commands natively for the project, improves WSL disk compatibility when the whole setup is doing native linux filesystem calls, and it also separates project dependencies of you main system. As in you can run whichever version of Node.js, Python for each project dev container without any conflicts with versions installed on your host OS. That way if project x recommends Debian, Python 3.10, Cmake 3.8 or whatever - no problem, just configure your devcontainer with those dependencies and you are good to go.

adam-ah commented 6 months ago

@MiikaH sorry, this is totally irrelevant. Could we please stay on topic and focus on the bug in WSL? (this message does not require a response)

MiikaH commented 6 months ago

Discussing workarounds for the bug is the only relevant use for this topic, so that people who are affected can find a working solution for their use case. Especially when it is clear that we may have to wait decade or two until Microsoft actually fixes the bug itself.

HugoImaios commented 6 months ago

You can't call "don't use X feature" a workaround for an issue titled "feature X does not work".

The issue is "File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps". Therefore, "don't use Windows filesystem, use the Linux one", does not seem an acceptable workaround for this particular issue (even if it actually works, as much as installing a full ubuntu, running a rapsberry or riding a bike)

WaldemarH commented 5 months ago

Nothing works :(

I've tried the /mnt/... version it doesn't work.

I've also tried the linux filesystem:

I've created /var/www/client folder
mounted "net use Y: \wsl.localhost\RockyLinux-8 /PERSISTENT:YES"
started webpack in client
edited a file on a Y disk inside the client folder and nothing, zero, nada, zilch

So currently WSL2 is useles for webpack development. :(

mcnilz commented 5 months ago

I found a pretty ugly workaround some time ago. Since the pain of this issue seems never to end I want to share it.

In my development docker image I added the "Unison file synchronizer", supervisor and scripts. Unison is used to synchronize the to /tmp directory (that must be volume or tmpfs but not bind). Supervisor is used to manage the ongoing sync process and other goodies out of the scope of the main issue.

The code for nodejs development is here: https://github.com/bridgefield/docker-node-dev Dotnet development here https://github.com/bridgefield/dotnet-core-sdk-extra The repositories are rarely updated sinice they are not in use at the moments. Maybe ppl can adopt the ideas to their projects.

The solution is far from being perfect. It takes much performance because unison can not use ionotify.

adam-ah commented 5 months ago

@WaldemarH maybe it's better to delete your not-so-relevant comments on this bug? This is a bug report about WSL2 file system change notifications, not a samba tutorial.

rw3iss commented 5 months ago

@adam-ah Laf dude... stfu. This bug has been open FOREVER. No one has a good solution, except some guy who spends weeks formulating one... and you have to bash it. Stupid problems sometimes require stupid solutions. Go bitch at Microsoft for their shitty kernel code. Thanks @WaldemarH ...nice workaround, albeit extensive.

WaldemarH commented 5 months ago

@rw3iss So should I put it back on? .. I know I would be happy if somebody gave me at least some working option.

rw3iss commented 5 months ago

@WaldemarH Yeah... would suggest just summarizing it a bit as you did, as an alternative solution, and then maybe linking to a gist with full instructions. Doesn't hurt and someone else may find it useful for sure. AFAIK it's one of the only ways to actually link them via network operations.

Fusseldieb commented 3 months ago

This is still an issue.

I have to use usePolling: true with vite to get it working, but this eats 60% of my CPU and just makes everything sluggish.

Will this ever be solved? We aren't talking about a solo maintainer here, but Microsoft.

nwoodr94 commented 3 months ago

My team and I have completed our migration to NextJS. About a dozen apps. No issues with HMR since.

On Thu, Jun 6, 2024, 6:59 AM Valentino Stillhardt @.***> wrote:

This is still an issue.

I have to use usePolling: true with vite to get it working, but this eats 60% of my CPU and just makes everything sluggish.

Will this ever be solved? We aren't talking about a solo maintainer here, but Microsoft.

— Reply to this email directly, view it on GitHub https://github.com/microsoft/WSL/issues/4739#issuecomment-2152374522, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK77FPBD3X37QYQBJ36U2S3ZGBMMPAVCNFSM4JWZQM4KU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJVGIZTONBVGIZA . You are receiving this because you commented.Message ID: @.***>

Fusseldieb commented 3 months ago

In the meantime I found a workaround using the interval parameter set to 1000 (1 second). The default is 100 and bombards the PC with requests. It's now down to 15% CPU usage on my 8th gen i7 machine.

Vite:

usePolling: true,
interval: 1000

It's now a lot faster, but it's also just a workaround, not a fix. Hope to see this fixed someday.

Fleker commented 3 months ago

Is there a way for Angular to use polling in its CLI? The most recent versions of Node and Angular no longer work on the Legacy WSL instance, which does have proper filesystem listening. While this bug was a bit tedious to start with, it is going to be increasingly hard to balance up-to-date dev tools in a Legacy container.

microsoft / WSL

[WSL2] File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps #4739

Bug report template