search

microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.21k stars 809 forks source link

[WSL2] File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps #4739

Open SteveSandersonMS opened 4 years ago

SteveSandersonMS commented 4 years ago

WSL2 is really close to being a perfect runtime environment for server apps being developed in Windows. Great job! One missing feature however is breaking a core part of the developer flow.

For sources stored on the Windows filesystem, any changes made by Windows applications such as Visual Studio do not trigger any file change notifications as far as Linux apps are concerned. This means that all "live rebuild"-type tools don't work (examples: webpack --watch, jekyll --interactive, and Tilt.dev) when running under WSL2. This unfortunately renders many modern dev workflows unviable.

Notes:

Bug report template

Finally I understand that the fix for this is likely to be "add file watch capabilities to the Plan9 server", and you may feel this is already being tracked by #4064. However #4064 describes a more obscure symptom of this missing feature and makes it sound like an intermittent issue. What I'm reporting here is not intermittent at all, and is a pretty mainstream scenario (using tools like webpack --watch). Thanks!

therealkenc commented 4 years ago

Yes, #4064 is most unfortunately titled. Worse, the OP has no repro steps and an unecessary symlink confusing the direction they are talking about. Craig's notepad.exe repro with cat isn't an inotify(7) (file watcher) problem, even if implementing inotify is the fix to whatever loosly implied sync problems are going on over there.

Nevertheless #4701 got closed as a dupe with (quoth):

We need to add file watch capabilities to the Plan9 server that serves files to a WSL2 distro, and we're tracking that work item here

So, best I can tell, #4064 is being treated as the LZ for inotify triggers from Windows to WSL2. For lack. #4169 is pretty much exactly your use-case also landed dupe #4064.

This did work great on WSL1

Yes, known regress.

nake89 commented 4 years ago

Is there a temporary fix to this? This is giving me a headache.

Using Windows 10, WSL 2. Running npm run serve on my Vue project, which normally hot reloads when I do changes (on my Linux and Mac, and WSL1 and I think maybe WSL2 before(?)). Now I have to shut down and restart the Vue server.

Will probably just install Linux on this machine. Developing on a Windows machine is a real pain.

craigloewen-msft commented 4 years ago

@nake89 are you able to move your project over to the Linux root file system? i.e: Store in your Linux home folder for example? Are there any factors that are blocking you from doing so?

And per @therealkenc 's comment this would be a much better landing zone for adding inotify to the 9P file server. I'll update my comment in #4701 to point here, and add some tags to this issue.

nake89 commented 4 years ago

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

SteveSandersonMS commented 4 years ago

@nake89 That's great if your scenario allows it. But just to clarify for anyone else reading, that's not a solution in general as it doesn't work for other Windows-based editors such as VS.

craigloewen-msft commented 4 years ago

@SteveSandersonMS agreed, we will still be tracking this issue here. :)

ghost commented 4 years ago

Hi, I am encountering this with docker-sync. My setup is:

If I make a change to a file directly inside Ubuntu, changes are reflected in the container's mounted volume. However, if I edit a file in PhpStorm in Windows, the change is reflected in Ubuntu as expected but not in the container.

I can't easily move my files to the Linux filesystem because I need to use PhpStorm and other Windows tools in my dev environment, and I also have other stuff such as Google drive sync running to back up local changes. So the workaround doesn't work for me.

Carl-Hugo commented 4 years ago

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

This is a great workaround that will save me lots of time until WSL2 supports this use-case. Thanks for sharing!

Carl-Hugo commented 4 years ago

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

This is a great workaround that will save me lots of time until WSL2 supports this use-case. Thanks for sharing!

I wrote a small blog post explaining this workaround and talking about a few more things that I discovered/experienced, using Jekyll on WSL2: Speed up your builds to up to 375% and watch for changes for an even faster dev cycle using this workaround on WSL2/Ubuntu, if that can help someone.

Note: that applies to more than just Jekyll, Jekyll it was just the catalyst.

safizn commented 4 years ago

For those stumbling upon, some notes about WSL2 features & limitations (OS build 19041.21, insiders slow ring):

ivellios commented 4 years ago

While I am having similar issue with my React App and I am looking for a solution to this as well, I can reply to:

I can't easily move my files to the Linux filesystem because I need to use PhpStorm and other Windows tools in my dev environment, and I also have other stuff such as Google drive sync running to back up local changes. So the workaround doesn't work for me. @weknowsoftware

@nake89 That's great if your scenario allows it. But just to clarify for anyone else reading, that's not a solution in general as it doesn't work for other Windows-based editors such as VS. @SteveSandersonMS

This hint by @myuseringithub:

  • Accessing Windows filesystem from WSL2, when developing is extremely slow. While moving projects to WSL2 filesystem, will increase performance, much faster than WSL1 & Windows development. (WSL2 can be accessed in path \\wsl$\)

is a great one to help you! Also I found out yesterday that since \\wsl$\ is a network resource in Windows you can easily map it to a drive. Then in ANY editor/tool you can use it as if it was on your computer. Just go to the Explorer and manually enter this address to access the resource. You will most likely see "Ubuntu" folder. It is a root of your WSL and it can be mapped to the drive (right click -> map to drive). That helped me setup my project on WSL natively, while being able to edit in the windows editor.

Though now I am struggling with Docker containers permissions, which I hope to solve separately and see if reloads in my app work.

Hope that helps.

nicks commented 4 years ago

Is there anything that app devs can do to workaround this in their apps? e.g., is there a different file-change notification API that would work on WSL2 across filesystems? (there are a lot of different file-change notification APIs). Or should we wait patiently for inotify support?

Carl-Hugo commented 4 years ago

Is there anything that app devs can do to workaround this in their apps? e.g., is there a different file-change notification API that would work on WSL2 across filesystems? (there are a lot of different file-change notification APIs). Or should we wait patiently for inotify support?

You can use the Linux file system directly, accessible from Windows too; see https://github.com/microsoft/WSL/issues/4739#issuecomment-582196872

mattlacey commented 4 years ago

@Carl-Hugo Does that work for you, as in, are changed detected? I've been using the \wsl$ path with windows editors for a while because with WSL2 the Linux FS is so much faster, but even when using windows editors with files stored there it's not triggering HMR when using npm watch.

Carl-Hugo commented 4 years ago

@mattlacey Yes it works...

  1. image
  2. ctrl+s the README.md file in VS Code (Windows)
  3. image

Side note: for-the-new-order is not a sect but a Star Wars RPG thing 😉

vielhuber commented 4 years ago

Some graphical programs (e.g. SmartGit/GitKraken) need to be installed in WSL2 and accessed through GUI client through Windows (Unix X server), to overcome the inotify events & performance degredations.

Can you please elaborate on that: Is this working well? Any tutorials / starting points how to set this up?

safizn commented 4 years ago

@vielhuber I wrote it as a comment for myself, tried to implement it once without success. Just wait till WSL2 will support inotify. You could still use graphical interfaces for git, only that you would have to constantly refresh to see changes, which is the same case when dealing with VSCode's git panel.

But if you wish to dig deeper, check out:

I wrote these comments when setting up my development environment

vielhuber commented 4 years ago

@myuseringithub Awesome thank you.

huysentruitw commented 4 years ago

I use my regular Git client in Windows, because my project is still on my Windows machine. However, to get livereload working, I simply add a symlink from my WSL2 home directory to the project on my windows machine, f.e.:

ln -s /mnt/d/projects/contrib/create-your-future-website/ ~/create-your-future-website

After that, I start jekyll serve --livereload from ~/create-your-future-website and open that folder in VS code using ctrl-shift-p and then type "Remote-WSL: New Window" as explained above.

This way, you don't have to move your project and can still enjoy your favorite git client in Windows. Profit!

huysentruitw commented 4 years ago

Arg, scratch the above idea. It only seems to get triggered by file changes at the top-level directory. 😢

aaronadamsCA commented 4 years ago

Oddly enough, yesterday I found that changes I made in the top-level directory triggered Gatsby hot reload. It's only in subdirectories that it wasn't working for me, which (eventually) led me to this ticket.

I certainly wasn't using any attempted workaround; this was out-of-the-box behaviour.

For example, editing /gatsby-config.js triggered a hot reload. Whereas editing /src/pages/index.js did not. Based on this ticket description, I wouldn't have expected either one to work. 🤷

rodrigogs commented 4 years ago

I'm very confused about this problem, there's a whole bunch of abandoned issues specifically about it. I already saw that movie with WSL 1 and it took me a hell of a time to figure it out.

mattbell87 commented 4 years ago

This issue also affects when you run Docker for Windows on Windows Home Edition and your only option is the WSL2 based engine!

To be specific I spun up a container that monitors LESS files and compiles them in to CSS when there is any change. But nope unfortunately nothing gets notified when files are changed. Also I know it's a workaround but I'd rather not move the project in to the Linux filesystem, the less I have to put in the readme file the better!

gsarig commented 4 years ago

Same here... I was looking forward to try WSL2 and when I installed it I bumped into this issue, so at least for now, I reverted back to WSL1.

BenMorel commented 4 years ago

Same here. I've been waiting for Windows 10 version 2004 for a long time to be able to use WSL2, just to discover that it's useless because of this issue. Back to VMWare.

flamedfury commented 4 years ago

Glad that I found this issue. Have been having hell of a time trying to understand what was going on with my new 2004 update / WSL 2 setup and my Hugo site. I must say not having reload while developing has been a great interruption to my workflow 😂.

laudukang commented 4 years ago

wsl2 + vs code, project file @ D disk

same problem

hariadiarief commented 4 years ago

I use my regular Git client in Windows, because my project is still on my Windows machine. However, to get livereload working, I simply add a symlink from my WSL2 home directory to the project on my windows machine, f.e.:

ln -s /mnt/d/projects/contrib/create-your-future-website/ ~/create-your-future-website

After that, I start jekyll serve --livereload from ~/create-your-future-website and open that folder in VS code using ctrl-shift-p and then type "Remote-WSL: New Window" as explained above.

This way, you don't have to move your project and can still enjoy your favorite git client in Windows. Profit!

it's still doesn't work for me, the source control still not showing update changes

fenghuang85 commented 4 years ago

Is there a temporary fix to this? This is giving me a headache.

Using Windows 10, WSL 2. Running npm run serve on my Vue project, which normally hot reloads when I do changes (on my Linux and Mac, and WSL1 and I think maybe WSL2 before(?)). Now I have to shut down and restart the Vue server.

Will probably just install Linux on this machine. Developing on a Windows machine is a real pain.

I use wsl command to set wsl version to 1, it resolves issue while you wait for the solution.

hariadiarief commented 4 years ago

I think for a temporary solution, you can move your repo/project to the inside WSL 2 directory.

For me it's solved the problem

image

barthicus commented 4 years ago

@hariadiarief @laudukang @gsarig @BenMorel @rodrigogs

I think for a temporary solution, you can move your repo/project to the inside WSL 2 directory.

For me it's solved the problem

image

I can confirm it finally works on WSL2! 🎉 Just moved my Next.js (React) project from /mnt/d (Windows 10 D:/ drive) to \\wsl$ directory and reloading is working. Moreover - everything is pretty damn fast . It's like on native Windows with this difference that now we have all ubuntu's juice 💪

Small tip for windows users:

And don't forget to open project in vscode using remote-WSL mode: ubuntu

This info should be pinned on every Docs that mention about developing on Windows under WSL.

I'm really glad I found this gh issue. I was trying to fix this problem for days and was pulling hairs from my head to just give up and leave WSL. Now that's awesome dev experience 😃

mateusnroll commented 4 years ago

I think i've got a workaround too, which lets you use it with any editor.

  1. Have the project files under the WSL filesystem, not windows (~/projects for example)
  2. Open a WSL shell, cd into the project directory and run explorer.exe . -- it will open the current WSL folder on windows explorer. As far as I could see, it just mounts the fs as a network device, so it should be safe as it does not seem to bypass anything.
  3. Open your project folder on any editor you'd like. SublimeText is working fine for Rails and Jekyll with live reload.

The only thing you loose is the ability to use some stuff, like GitHub Desktop (I could not get it to work, but git cli is fine too). Other than that, it's been pretty smooth.

bitstarr commented 4 years ago

Why is this issue labelled 'feature'? This is a bug and it's known over half a year now! I have my development data on a separate drive, so it won't be lost if something happens to my windows system. That's why the suggested temporary fix is impractical to me.. It would be nice if it will be fixed ASAP so people can separate environments and their data.

jasonswearingen commented 4 years ago

For me the better workaround is to use WSL1, you can learn to use the wsl app via commandline to have/manage multiple ubuntu vm's if you go this route.

jasonswearingen commented 4 years ago

sigh WSL 1 has other issues, such as #2448 which doesn't look like it was ever fixed (for wsl1)

jasonswearingen commented 4 years ago

here is my current workaround, doing development via VSCode: 1) code in a git repo, windows-side (eg: c:/repos/proj) 2) create a WSL2 vm,
3) create a linux side clone of the repo using SourceTree (eg, clone into \\wsl$\vmName\repos\proj).

For whatever reason pushing/pulling changes between the windows and WSL vm's via git is faster than copy/paste files via windows explorer. This workflow ensures that my code isn't destroyed if I accidentally delete the WSL vm.

hope that helps someone.

BlackGlory commented 4 years ago

I tried converting the code repositories on the Windows side to bare repositories, and then setting them as git remotes on the Linux side.

But when you have a lot of repositories, you need to make sure that the configuration files for each repository are configured correctly. And you need to make sure you're pushing to multiple upstreams (both local and remote) with each push, which is a bit cumbersome for command-line git.

I finally had to abandon this approach and just create a cron task that uses rsync to synchronize code from the Linux side to the Windows side every hour. This also works with VSS-based backup solutions (VSS cannot backup WSL2).

aaronadamsCA commented 4 years ago

To anyone still struggling with this, consider using Remote-Containers to load your repo directly in a container. While the extension is still in "preview" it's quite mature now (aside from the docs), and at this point I can't see any reason I'd ever want to check out any Linux app in my Windows filesystem anymore.

You can create a .devcontainer/devcontainer.json as short as this, if there's a suitable MCR image for you:

{
  "image": "mcr.microsoft.com/vscode/devcontainers/javascript-node:12"
}

If you want more control, Remote-Containers: Add Development Container Configuration Files will give you lots of starter environments for all kinds of projects:

image

It creates a simple Dockerfile you can edit to install your own packages:

#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------

# To fully customize the contents of this image, use the following Dockerfile instead:
# https://github.com/microsoft/vscode-dev-containers/tree/v0.128.0/containers/javascript-node-12/.devcontainer/Dockerfile
FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:0-12

# ** [Optional] Uncomment this section to install additional packages. **
#
# RUN apt-get update \
#     && export DEBIAN_FRONTEND=noninteractive \
#     && apt-get -y install --no-install-recommends <your-package-list-here>

With this in place in my repository, I've got the following instructions to get going:

Recommended environment: Docker + Visual Studio Code

You can open this repository as a container in Visual Studio Code:

  1. Install Remote - Containers extension
  2. Run command Remote-Containers: Open Repository in Container...
  3. Enter repository name user/repo
  4. Open terminal and run npm install

To configure Git, fix any incorrect values in ~/.gitconfig imported from your local Windows system, and set your container username/email:

git config --global core.editor "code --wait"
git config --global core.autocrlf input
git config --global --unset core.sshCommand
git config --global user.name "Aaron Adams"
git config --global user.email aaron@aaronadams.ca

Your mileage may vary, but I haven't found any drawbacks to this approach, whereas the 10x increase in filesystem speed was a boon to my workflow. This issue still needs fixing, but a development container leveraging the speed of the WSL2 filesystem is the better way to go 🙂

jasonswearingen commented 4 years ago

@BlackGlory that's why I use SourceTree, it can automatically push upstream on a commit (though I think you can configure git command-line to do it too... hooks or something)

@aaronadamsCA Thanks for mentioning this. It seems like my git based workaround (above post) is getting pretty close to that, logically.

ryan7273 commented 4 years ago

Some of us have reasons for not moving everything into the Linux filesystem. For anyone else with that problem, my workaround was to create an empty file in the container and include it in the list of files being watched by nodemon. There are several extensions available that run a script on file save, so I installed one and have it touch the empty file any time any of the watched files gets updated.

jasonswearingen commented 4 years ago

@ryan7273 I agree and wish this bug got fixed.

@aaronadamsCA FYI your workaround requires an external repository. If you are hacking stuff together thats pretty annoying to do. plus you can't use gui based diff tools like sourcetree to manage your checkin.

kevinvalk commented 4 years ago

Same here, just want to keep my files together in Windows land and host (dev) it using Linux tooling. This always worked great with WSL and Docker for Windows Hyper-V. However, with WSL2 this does not work anymore....

I do not want to move my files to the Linux root system as I have a backup solution that synchronizes my whole User folder and everything under that. If I move some "important" files to the Linux file system I have to keep track of this and back those files up as well.

So for me, moving the files to Linux is not a solution and I would really like to have it work like WSL1 (inotify events from Windows -> Linux).

CaribouJohn commented 4 years ago

I am having this issue it seems with a vscode extension inside a dev-container

See https://github.com/microsoft/vscode-remote-release/issues/3491

My issue is with a mounted folder in the dev-container which is the use case for the dev-container IMHO.....

relston commented 3 years ago

Just keeping this thread alive. This is one of those quality-of-life issues that may not seem critical but, when you run into it every day, really detracts from the polish of WSL

gsarig commented 3 years ago

Just keeping this thread alive. This is one of those quality-of-life issues that may not seem critical but, when you run into it every day, really detracts from the polish of WSL

Maybe it's not critical, but for me, at least, it's the reason I'm still stuck with the old WSL.

kerryfalk commented 3 years ago

Has MS committed to making this change? I'm considering going back to WSL1 as it was working quite well. In the WSL1 to WSL2 comparison MS states:

https://docs.microsoft.com/en-us/windows/wsl/compare-versions

"We recommend against working across operating systems with your files, unless you have a specific reason for doing so. For the fastest performance speed, store your files in the WSL file system if you are working in a Linux command line (Ubuntu, OpenSUSE, etc). If your working in a Windows command line (PowerShell, Command Prompt), store your files in the Windows file system."

also:

"We recommend that you use WSL 2 as it offers faster performance and 100% system call compatibility. However, there are a few specific scenarios where you might prefer using WSL 1. Consider using WSL 1 if:

Your project files must be stored in the Windows file system. WSL 1 offers faster access to files mounted from Windows. If you will be using your WSL Linux distribution to access project files on the Windows file system, and these files cannot be stored on the Linux file system, you will achieve faster performance across the OS files systems by using WSL 1. A project which requires cross-compilation using both Windows and Linux tools on the same files. File performance across the Windows and Linux operating systems is faster in WSL 1 than WSL 2, so if you are using Windows applications to access Linux files, you will currently achieve faster performance with WSL 1."

jasonswearingen commented 3 years ago

@kerryfalk that's a great article, wish it was out a few months ago when I started my WSL investigations. From my hard fought experience I agree with it's suggestions

Soviut commented 3 years ago

A workaround for anyone using Webpack or other tool that uses Chokidar under the hood; you can set an environment variable to use polling instead of listening for file events.

CHOKIDAR_USEPOLLING=1

I use this with Docker by putting it in a .env file that Docker Compose picks up when it starts. However, you should be able to prepend any start commands with this env as well.

CHOKIDAR_USEPOLLING=1 npm start

This isn't a great solution since it dramatically increases the resources the OS or Docker needs to consume in order to perform the polling and it will drain your laptop batteries way faster than actual file system event watching, but it works until WSL2 triggers those file system events.

helgatheviking commented 3 years ago

Following. Gotta go back to WSL 1 for now.

AlbertMarashi commented 3 years ago

Need this so bad..