microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.25k stars 812 forks source link

[WSL2] File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps #4739

Open SteveSandersonMS opened 4 years ago

SteveSandersonMS commented 4 years ago

WSL2 is really close to being a perfect runtime environment for server apps being developed in Windows. Great job! One missing feature however is breaking a core part of the developer flow.

For sources stored on the Windows filesystem, any changes made by Windows applications such as Visual Studio do not trigger any file change notifications as far as Linux apps are concerned. This means that all "live rebuild"-type tools don't work (examples: webpack --watch, jekyll --interactive, and Tilt.dev) when running under WSL2. This unfortunately renders many modern dev workflows unviable.

Notes:

Bug report template

Finally I understand that the fix for this is likely to be "add file watch capabilities to the Plan9 server", and you may feel this is already being tracked by #4064. However #4064 describes a more obscure symptom of this missing feature and makes it sound like an intermittent issue. What I'm reporting here is not intermittent at all, and is a pretty mainstream scenario (using tools like webpack --watch). Thanks!

asteinarson commented 3 years ago

I stumbled on this problem when working with a Ruby project in WSL2 (with WSL extension in VS code). I first noticed that the source control panel did not update on saving files. Then, that debug configurations (edits to launch.json) only were refreshed after restarting VS code. Finally the same things with tasks... so I had to investigate.

Moving my project files to the Linux side was not an option (I run Sketchup on the Windows side, consuming the Ruby files). But I still need a snappy development environment, with Linux/WSL capabilities.

Sadly I realized there are (currently) no simple fixes to the broken inotify functionality of the mounted 9P windows file system (I tried a number of things, including watching the directory inside of Linux and injecting change events there). No easy way forward... back in cross platform murky land...

My solution in the end looks like syncthing - a versatile directory sync tool - over TCP. So I have two copies of the project, one on Windows side and one on Linux side. And they are kept in sync, and both behave as local file systems (because they are).

Took me a day to stumble, research, learn about the issue and experiment with this solution...

Would be nice to have this working out of the box with WSL2.

BenMorel commented 3 years ago

@asteinarson I had a similar setup for a while with Unison, but had to give it up because it would more often than not crash when changes occurred on both sides (IDE + git on Windows, web server + CLI tools w/ file watchers on Linux VM).

I hope you'll be luckier!

Arephan commented 3 years ago

npm run serve from git bash. It will hot reload.

johnjelinek commented 3 years ago

bump

ManfredLange commented 3 years ago

The issue can be avoided by not mixing file systems, e.g. NTFS with Linux file system until a permanent fix is available by Microsoft for WSL2.

To help others saving time and pain, I wrote up my findings in an article. In it I also describe how I resolved the issue for the combination of VSCode, WSL2 and a Linux-based dev container for .NET. Here is the link: https://levelup.gitconnected.com/docker-desktop-on-wsl2-the-problem-with-mixing-file-systems-a8b5dcd79b22?sk=53d24e33a9f247fd626e3aa6959de7d4

arialpew commented 3 years ago

My solution to have everything work seamlessly is to downgrade to WSL1.

Do Not Upgrade to WSL2.

Going back to WSL1 ? Open a PowerShell :

wsl -l -v
wsl --set-version <distro-name> 1

Even if your favorite program advocate for it (like Vscode who tell you WSL2 have better performance), there's so many pitfall that it's not worth doing it.

File system is broken, you can't work in /mnt/c or /c/, you have to put all of your workflow into the Linux File System (that mean you can't access it with Windows app like GitKraken). And nothing seem to evolve.

Edit : I don't want to be to critic because I really like WSL, it's a game changer project. It's not bad to wait things stabilized before upgrading to WSL2. Homewever, I revert what I said about the /home/ directory, it seem it's a best practice to put your files here for better performance and file system seem to work fine. You can still access your files on Windows app with "\wsl$" path, it's a network file system. But anyway, mixing file systems is always a complicated issue. I hope this issue will get more traction in 2021 👍 .

https://devblogs.microsoft.com/commandline/wsl-2-is-now-available-in-windows-insiders/

mindplay-dk commented 3 years ago

File system is broken, you can't work in /mnt/c or /c/, you have to put all of your workflow into the Linux File System (that mean you can't access it with Windows app like GitKraken). And nothing seem to evolve.

This. I don't understand why we have to pick between decent file-system performance under Linux, or a normal, practical, functioning everyday life with Windows. These shouldn't be mutually exclusive options.

If the file-system performance in Windows isn't up to the task, really what they should be doing is improve the native Windows file-system performance - which would surely be a win, not just for Linux usage.

If the file-system facilities in Windows can't handle the Linux permission data, maybe it's time to enhance the file-system with better meta-data capabilities.

If the file-systems are too different to integrate, maybe it's time to add support for native Linux file-systems in Windows.

And so on.

Having to use a virtual disk seems like a tedious workaround - I'm really hoping we'll someday soon see real improvements to the underlying technology, instead of these half measures. Four years on and WSL remains this "would-be awesome" feature that seems to always be "almost there", but never quite gets there.

Damnit, Microsoft, we're ready - now take us there, please! 🙏

craigloewen-msft commented 3 years ago

@kMeillet and @mindplay-dk that's some overall great feedback! I agree completely, one of the biggest challenges for any user upgrading to WSL 2 is the fact that they will have to put their projects into the Linux file system to gain the performance advantages. On the team we're investigating ways to improve this experience (and always open for different ideas!). One of the major factors here is that currently the maximum speed we could theoretically make WSL 2 run on the /mnt/c/ file system is the same speeds as WSL 1, which is still too slow for many users. Overall I believe this speaks to the points in this thread, and we are looking into plans here that are going to a be a mixture of long term and short term improvements.

denisvmedia commented 3 years ago

Honestly, the suggestion to use linux filesystem is killing me. Why? Because the only filesystem you can trust is your real machine filesystem. I manage my folders and files and I know what I'm doing with them. But I don't manage the volume in WSL2 (I even didn't create it - it was "magically" created with the system upgrade and activation of WSL2). So, personally, I'm afraid that due to some kind of my ignorance I risk to delete it occasionally. Maybe to some of you it will sound silly/not enough persuasive, but for me switching to this way of workflow is almost not anyhow different from running just a virtual machine with a mounted folder. With the slight bonus for a VM due to the fact that I do there everything explicitly.

eduter commented 3 years ago

One of the major factors here is that currently the maximum speed we could theoretically make WSL 2 run on the /mnt/c/ file system is the same speeds as WSL 1, which is still too slow for many users.

Personally, I would love to have WSL 2 with the same performance of WSL 1, but usable. As it is, it's just not usable to me. With WSL 2 I would be able to tell my IDE (JetBrains) to use Git from the Linux VM, making Husky hooks work (Linux binaries don't work when using Git for Windows). Unfortunately, this feature only supports WSL 2.

mindplay-dk commented 3 years ago

One of the major factors here is that currently the maximum speed we could theoretically make WSL 2 run on the /mnt/c/ file system is the same speeds as WSL 1, which is still too slow for many users.

How about adding a write-behind cache buffer of some sort? Even a few megabytes should help tremendously, as the performance problems are mostly felt with things like npm install, compilers etc., which creates thousands of tiny files.

I understand this would make the guest OS less resilient to accidental shutdown or a hard crash - but I think, mostly, Linux systems on WSL are used for development, where someone would likely prefer performance over some theoretical risk of data loss under some unusual circumstances? (I've never heard of anybody running production services on WSL? You could of course provide a way to disable a cache feature.)

fenghuang85 commented 3 years ago

One of the major factors here is that currently the maximum speed we could theoretically make WSL 2 run on the /mnt/c/ file system is the same speeds as WSL 1, which is still too slow for many users.

How about adding a write-behind cache buffer of some sort? Even a few megabytes should help tremendously, as the performance problems are mostly felt with things like npm install, compilers etc., which creates thousands of tiny files.

I understand this would make the guest OS less resilient to accidental shutdown or a hard crash - but I think, mostly, Linux systems on WSL are used for development, where someone would likely prefer performance over some theoretical risk of data loss under some unusual circumstances? (I've never heard of anybody running production services on WSL? You could of course provide a way to disable a cache feature.)

That's a very good point, at least, I don't really care about data loss unless it break my system

didix16 commented 3 years ago

So any news about this bug fixing? It's very annoying moving the code to linux filesystem to properly develop...

ignacioanezin commented 3 years ago

I just want VS Code Live Server extension to work correctly on mnt\c with wsl2. Downgrading to version 1 does not feel like a good workaround.

dejayc commented 3 years ago

Try prefixing your WSL2 command invocations with the following environment variable, if the commands need to subscribe to filesystem I/O notifications:

CHOKIDAR_USEPOLLING=true
Soviut commented 3 years ago

@dejayc this will work but it comes with a huge performance cost. CPU usage will increase significantly (30% on my system) as it needs to do a lot of active polling of the file system for changes. Additionally, the change detection is not nearly as fast as on the pure linux file system.

bobarros commented 3 years ago

This should be a huge warning before allowing any upgrade. A lot of effort to migrate to WSL2 and now I have to make my way back. So much time was lost trying to understand what was happening for no reason.

IanCirrus commented 3 years ago

I am developing code which needs to run on both Windows and Linux. My primary development environment is Windows, but sometimes it is useful to be able to build and test the same code in a Linux environment during development.

I have a symlink in my WSL environment to my main dev code area in Windows. I have just built in WSL2 and found a GCC warning I don't care about in a boost header, so I updated my source to disable that warning while including the header. The build still failed.

After a bit of head-scratching, I thought to do an ls -l on the file under WSL, and noticed the last-modified time was still yesterday afternoon, but the symlink is definitely pointing to the file I just modified.

Putting my development tree in the WSL filesystem and linking from Windows makes NO sense to me.

I consider this a serious bug in WSL2. Fortunately I found this thread and downgrading to WSL1 fixes the problem, but this is a deal-breaker for me with WSL2.

mindplay-dk commented 3 years ago

With the recently-announced official support for GUI apps, this seems now even more important?

Since this opens WSL to non-developers, who just want to run Linux desktop software on their PC, aren't they going to run into weird situations when they try to work with files and documents in their PC home folders? 🤔

jasonswearingen commented 3 years ago

If you use VSCode, this workaround may be useful: I have switched to container based development. So, all your work files are located on the container, not on the windows "native" partition. The key which made this workaround useful for me is that you can still access the container source files via windows explorer as a psudo-network share. I can still use SourceTree (a windows, GUI based git client) by passing it the git repo location. Having a gui based git-client was the main reason I wanted my source files on the windows file system, so this workaround effectively solves everything for me.

If you want to access your docker containers files (mounted partitions) from windows, it is located at something like: \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\vsc-remote-containers\_data This location is very poorly documented and a lot of false information floating around at places like stack overflow. only by luck did I stumble on it.

I hope that helps you out too! Again, This is useful if you need to access the 'source' filesystem from windows while needing fast+fs notify support. You do need to watch out for filesystem permissions though, as in my case, my first attempt resulted in a slightly corrupt git folder because permissions defaulted to ROOT/ROOT. After I ensured the folder permissions include the linux user's user/group and re-downloaded the repository everything worked fine (and no git corruption has occured)

Edit: a big bonus side effect is that with vscode, the linux container's definition is now included in the git repository under a .devcontainer folder, ensuring that the entire dev stack is now reproducable using the git repo as the single source.

bobarros commented 3 years ago

If you use VSCode, this workaround may be useful: I have switched to container based development. So, all your work files are located on the container, not on the windows "native" partition. The key which made this workaround useful for me is that you can still access the container source files via windows explorer as a psudo-network share. I can still use SourceTree (a windows, GUI based git client) by passing it the git repo location. Having a gui based git-client was the main reason I wanted my source files on the windows file system, so this workaround effectively solves everything for me.

If you want to access your docker containers files (mounted partitions) from windows, it is located at something like: \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\vsc-remote-containers\_data This location is very poorly documented and a lot of false information floating around at places like stack overflow. only by luck did I stumble on it.

I hope that helps you out too! Again, This is useful if you need to access the 'source' filesystem from windows while needing fast+fs notify support. You do need to watch out for filesystem permissions though, as in my case, my first attempt resulted in a slightly corrupt git folder because permissions defaulted to ROOT/ROOT. After I ensured the folder permissions include the linux user's user/group and re-downloaded the repository everything worked fine (and no git corruption has occured)

Edit: a big bonus side effect is that with vscode, the linux container's definition is now included in the git repository under a .devcontainer folder, ensuring that the entire dev stack is now reproducable using the git repo as the single source.

Thanks for sharing this, @jasonswearingen

craigloewen-msft commented 3 years ago

@mindplay-dk WSL overall is targeted towards developer audiences, GUI app support in WSL is targeted towards allowing developers to use Linux GUI apps on Windows.

The feedback in this thread is super valuable to us. I understand that many folks can't move their files over to the Linux file system, and it's helpful for us to see the exact whys listed above. As @jasonswearingen detailed there are some great workflows now for projects living in the Linux file system (VS Code Remote, and now GUI apps allows you to access these projects more easily). I'll make sure to keep this thread updated with any news here, thank you!

asteinarson commented 3 years ago

For me, the use use case I look for is to be able to have a file open in a Windows app (usually an editor) and as that file is updated (often by tooling) on the Linux side, I 'd like to see that in the Windows app (without manually going there and refreshing the window / rescanning the directory).

It holds for MS file explorer as well, I'd like it to show accurate file sizes and file dates, when I browse files that are located on the Linux side.

A particular use case I had was developing Ruby Sketchup extensions. Now, Sketchup can only run in Windows environments, so my plugin code needs to be available on that system. However, the tooling I use (to edit, to generate the Ruby code), I prefer to run in a good Linux environment. I was not able to set that up in a straight forward way (I ended up with something like a deploy, from Linux to Windows).

I don't expect "stellar file system performance" in these scenarios. It's more about seeing changes done to a handful of files (on the Linux side) being accurately reflected on the Windows side, within a second or two.

pwnreza commented 3 years ago

If you use VSCode, this workaround may be useful: I have switched to container based development. So, all your work files are located on the container, not on the windows "native" partition. The key which made this workaround useful for me is that you can still access the container source files via windows explorer as a psudo-network share. I can still use SourceTree (a windows, GUI based git client) by passing it the git repo location. Having a gui based git-client was the main reason I wanted my source files on the windows file system, so this workaround effectively solves everything for me.

If you want to access your docker containers files (mounted partitions) from windows, it is located at something like: \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\vsc-remote-containers\_data This location is very poorly documented and a lot of false information floating around at places like stack overflow. only by luck did I stumble on it.

I hope that helps you out too! Again, This is useful if you need to access the 'source' filesystem from windows while needing fast+fs notify support. You do need to watch out for filesystem permissions though, as in my case, my first attempt resulted in a slightly corrupt git folder because permissions defaulted to ROOT/ROOT. After I ensured the folder permissions include the linux user's user/group and re-downloaded the repository everything worked fine (and no git corruption has occured)

Edit: a big bonus side effect is that with vscode, the linux container's definition is now included in the git repository under a .devcontainer folder, ensuring that the entire dev stack is now reproducable using the git repo as the single source.

Thx for sharing. I will check this out

dejayc commented 3 years ago

@dejayc this will work but it comes with a huge performance cost. CPU usage will increase significantly (30% on my system) as it needs to do a lot of active polling of the file system for changes. Additionally, the change detection is not nearly as fast as on the pure linux file system.

Using CHOKIDAR_USEPOLLING=true on my relatively modest laptop doesn't increase CPU usage by even 1%.

Soviut commented 3 years ago

Using CHOKIDAR_USEPOLLING=true on my relatively modest laptop doesn't increase CPU usage by even 1%.

It depends significantly on the number of files being watched. On my powerful desktop workstation CPU usage would rise sometimes to 50% depending on which project I have open.

rlabrecque commented 3 years ago

One of the major factors here is that currently the maximum speed we could theoretically make WSL 2 run on the /mnt/c/ file system is the same speeds as WSL 1, which is still too slow for many users.

Just to add my two cents; while I do agree that it would still be too slow for many users, getting back into that range of being some 10x faster would probably be 'enough' for most of us. It wasn't a problem I personally had with WSL1, but it is a very real problem with WSL2 unfortunately. I'm able to move some of my projects over to the wsl side but not all of them, two reasons for this are:

  1. One of the project is massive. It has to sit on a secondary drive which is larger than my primary drive. Given that I don't manage the wsl2 volume I worry about this. It may be possible to change it's location but we're just shuffling the problem around. Not to mention how would I even copy it in, do I need a 3rd drive as a buffer? Do I need to do it in chunks? Thinking about potential workarounds just fills me with dread.
  2. Aspects of the projects need to be more-usable from Windows than from Linux. I work in a very split development environment and at the end of the day we'd rather everything work better on just Windows than on just Linux. We have a client application and backend service application in the same repo, our windows editors like Visual Studio 2019, our windows tools like Git Fork and P4V need to play well with our environment. It's unfortunate that trying to run our backend with WSL2 experiences an unusable slowdown and this issue compared to running via tools such as "git bash" or WSL1.

It may be worth providing an official update over in #4197 since it's been over a year since the last update, and the discussion has gotten very fragmented with that issue locked. Or maybe just a meta discussion issue for these various WSL1 -> WSL2 filesystem regressions?

rw3iss commented 3 years ago

Well, tried all manner of things to get this working under Windows + WSL 2... still cant. Followed this blog post too: https://www.forevolve.com/en/articles/2020/02/04/speed-up-your-builds-and-watch-for-changes-to-up-to-375-percent-using-this-workaround-on-wsl2-ubuntu/

I tried hosting original project in Windows file system, and Ubuntu file system. I've tried starting VSCode from both Windows and WSL. Have tried VSCode in Remote-WSL mode, and regular. I've tried running my server from both WSL Terminal, as well as the VSCode built-in Terminal.

None of that is working/getting file changed events. Wtf Windows. Back to Ubuntu native. Grumble.

affans commented 3 years ago

Is there a timeline for when this is going to be implemented/fixed? I make heavy use of OneDrive with all my scientific projects/code files hosted there . It's not really feasible for me to break up my directory structure and start putting files in WSL2.

olegKusov commented 3 years ago

When it will be fixed? I can't use Sublime 4 (file tree doesnt update)

pwnreza commented 3 years ago

I know that I am not adding a fix and that many people probably know about this but in case someone who has such problems is struggling because of difficult access to the filesystem of the OS running in WSL:

One can access the filesystem of the OS via Network connection

All currently running distributions (wsl -l) are accessible via network connection. To get there run a command [WIN+R] (keyboard shortcut) or type in File Explorer address bar \wsl$ to find respective distribution names and access their root file systems.

https://docs.microsoft.com/en-us/windows/wsl/compare-versions#performance-across-os-file-systems

Nevertheless, I would love to use my windows filesystem in my linux distro in WLS2 with full capabilities.

tylercasper commented 3 years ago

Is this being worked on? This issue renders many dev environments (and thus WSL as a whole) mostly useless for me and undoubtedly others.

The solution of prepending CHOKIDAR_USEPOLLING=true to the pertinent command doesn't work in the case of more complex applications, such as running VS Code.

shazada commented 2 years ago

Same, any update on this? I use Eclipse with a Java application, through WSL2 and docker volumes shared I'm working on the Windows end the files aren't hot-reloading.

Any fix on eclipse and Java maybe from others? No issue with Hyper-v though (just a lot slower)

felipe-linares commented 2 years ago

Any update or workaround for this? Looking to use https://github.com/cosmtrek/air for some docker containers with the code in windows but docker running WSL2 backend. No file updates from windows and therefore air is not hot reloading...

raffaeler commented 2 years ago

I add the rsync scenario developing with .NET:

piaoger commented 2 years ago

In my scenario, I want to share directory in windows and watch file changes in /mnt/d. Maybe I have to do the whole job all inside Linux.

jeremyVignelles commented 2 years ago

Am I right to think that this issue is blocked by the lack of inotify support in FUSE ? Does WSL2 use FUSE under the hood ?

nzbr commented 2 years ago

If I am not mistaken, it uses the 9P file system over VSOCK. That does not rely on FUSE; it is a linux kernel module

floyd-may commented 2 years ago

This paints me into a corner where I have to keep my app on an older version of Parcel that (optionally) uses a polling-based approach to detect file changes. The older version lets me edit files in VS Code on Windows, and the changes get picked up in a running docker container (on WSL2). The detection isn't awesome or quick, but it works. My bigger concern is that I'm stuck with old versions of libraries that aren't being maintained anymore.

Dacesilian commented 2 years ago

I just adding my comment that I have also this problem and it means that I can't use Docker for developing my web app :(

jankap commented 2 years ago

This issue might be the reason, why remote development in VSCode does not work well on Windows. All tools that rely on the state of the file, like CMake, or the Git extension need manual refresh each time a file has changed. That's extremely annoying.

Will this problem be tackled?

Thanks :)

qdm12 commented 2 years ago

@jankap https://levelup.gitconnected.com/docker-desktop-on-wsl2-the-problem-with-mixing-file-systems-a8b5dcd79b22 should fix it although I haven't tried yet.

Also adding to this developing frontend via a development container with hot reload not working is terrible. It would be nice to have this fixed one day.

MFaiqKhan commented 2 years ago

yep, parceljs/webpack 5 hmr not working

mkoeppen commented 1 year ago

I have a node container and found an fix that works for me. I added the CHOKIDAR_USEPOLLING environment variable to the container in my docker-compose.yml. Now my watchers are working.


mynodecontainer:
    [...]
    environment: 
      - CHOKIDAR_USEPOLLING=1

Maybe it is not the best solution, but now i can develop with docker again.

bit-pax commented 1 year ago

I'm using Rollup and it can't detect changes even with polling.

amzon-ex commented 1 year ago

What is the status of this issue?

EmmanuelTheCoder commented 1 year ago

This issue persists. Is there a timeline for when it will be resolved?

jwleon commented 1 year ago

Also having this issue, whish it gets fixed but 2 years without progress make me doubt it...

shazada commented 1 year ago

BUILD TIME Comparism with Maven, Java + Docker when people tell me that WSL2 is faster when build from the linux subsystem..... well these numbers don't lie

- Powershell default setup (second time)

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25.722 s
[INFO] Finished at: 2022-06-02T18:57:23+02:00

- WSL2: Started build in WSL2 didn't move the data /mnt/c  (first time)

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:15 min
[INFO] Finished at: 2022-06-02T19:05:52+02:00

- WSL2: did a fresh git-clone in /home/git (first time)

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:12 min
[INFO] Finished at: 2022-06-02T19:13:50+02:00

- WSL2: did a fresh git-clone in /home/git (second time)

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:52 min
[INFO] Finished at: 2022-06-02T19:22:03+02:00

- WSL2:  fit-clone in /home/git & also clean maven artifacts in  /home/.m2

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 47.331 s
[INFO] Finished at: 2022-06-02T19:45:44+02:00

So all compared WSL2 is still almost 2x slower than build directly from Windows ==> Hence this BUG should get more priority I'm not even counting the extra time with the Remote VSCode and terminal time you loose initially connection and starting the WSL2.

potchy commented 1 year ago

Very disappointing that this is still an issue after 3 years.