search

microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.26k stars 812 forks source link

WSL 2 Ubuntu Files Gone #4974

Closed engina closed 3 years ago

engina commented 4 years ago

See our contributing instructions for assistance.

Please fill out the below information:

BTW after I killed the launch process, later I noticed bsdtar in process explorer.

What could be the problem? Could this be reversed? (I've checked if there is a backup of the rootfs dir in AppData to restore from there, at least retrieve but unluckily there was none)

willyjchen commented 4 years ago

I have a similar issue. My Ubuntu boot into a different distribution, which seems to have a new wsl2-linux-kernel.

.# cat /proc/version Linux version 4.19.84-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Wed Nov 13 11:44:37 UTC 2019

engina commented 4 years ago

I have a similar issue. My Ubuntu boot into a different distribution, which seems to have a new wsl2-linux-kernel.

.# cat /proc/version Linux version 4.19.84-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Wed Nov 13 11:44:37 UTC 2019

File system is stored in %APPDATA%/Local/Packages/CanonicalGroupLimited.Ubuntu18.04....

If you have File History enabled maybe you can save your files. I was not as lucky. I hope you are.

Meanwhile anyone figure out why is this happening?

sirredbeard commented 4 years ago

@engina

You are on WSL1, correct, on build 18362.657?

@willyjchen

What build are you on? Are you on WSL2?

shelmire commented 4 years ago

I also just experienced this. This is wild - I've never seen a system where the OS would reinstall itself and wipe files before. Exactly the same symptoms as OP. Truly incredible. Do I reinstall everything and trust Windows / WSL not to do something so incredibly awful again? Do I give up on WSL and go back to dual booting? Do I buy a personal Macbook Pro? Just wow.

engina commented 4 years ago

I also just experienced this. This is wild - I've never seen a system where the OS would reinstall itself and wipe files before. Exactly the same symptoms as OP. Truly incredible. Do I reinstall everything and trust Windows / WSL not to do something so incredibly awful again? Do I give up on WSL and go back to dual booting? Do I buy a personal Macbook Pro? Just wow.

Yes, there's a psychological component to this. I'll have real hard time trusting WSL again. I literally transferred all the tiny bit of dev work I've started doing on WSL (VS Code remote) to macs again. (Not that macs are perfect...)

WSL design should be more transparent, even though I've read almost everything I've found still it was a mystery where the storage files are (WSL1 is on host FS, WSL2 is on a vdd), at least that way we could have a way to backup etc.

It's ironic that this happened the day after I've praised WSL developers in a public forum, saying; they must be real proud of what they've been doing so far, it's a real promising development environment.

Next day, poof, everything is gone. :)

Anyway, I'm thankful for the WSL project nonetheless. Great work guys! (Except for the loss of several users' data :))

onomatopellan commented 4 years ago

Did you use the undocumented option wsl.exe --upgrade by any chance? That option has been removed in recent builds because it can blow up your distro. #4094

shelmire commented 4 years ago

@onomatopellan Absolutely not. I haven't touch my Ubuntu / WSL in months at least.

What's worse? This happened again this morning - I open up vscode, expect it to open wsl without issue since I just reinstalled it on saturday. But nope. Gone again, mysteriously, with no action on my end. Maybe something about Windows restarting is causing this? Could be some auto-update feature that I didn't successfully turn off?

onomatopellan commented 4 years ago

@shelmire woah, that's really bad. You should use Event Viewer or Task Scheduler to see latest run tasks and at what time they were done. Maybe some disk optimization program is deleting your rootfs folder.

craigloewen-msft commented 4 years ago

@shelmire what version of Windows are you using? We identified a possible cause of this bug earlier and have implemented a fix in Windows builds 19002 or higher. See this comment for details: https://github.com/microsoft/WSL/issues/4469#issuecomment-543401572

abdullahiphys commented 3 years ago

I have a similar issue. My Ubuntu boot into a different distribution, which seems to have a new wsl2-linux-kernel.

.# cat /proc/version Linux version 4.19.84-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Wed Nov 13 11:44:37 UTC 2019

Just faced this issue and this reply gave me some clue. I just opened Windows Terminal Settings file and saw an additional Ubuntu distro, which I did not have. I removed it and then my wsl2 ubuntu showed up in the list of windows terminal shell programs

mrmos commented 3 years ago

I'm having a similar issue where every time I close / re-open a terminal window, all of my files disappear.

bpulliam commented 3 years ago

Another possible source of this was fixed in Insider build 21370. In that case, we added a check during distro registration to better handle the case where the vhd file already exists and prevent it from overwriting the old copy. I understand that there may be existing versions of this issue, even with that fix, so I'm going to close this issue to ensure that existing cases get looked at appropriately.

Tehvan commented 3 years ago

Pretty sure I just had my files (just in the home directory?) disappear. Only change I can think of was that I installed Visual Studio Code and the Remote - WSL extension. Microsoft Windows [Version 10.0.19043.1052] Fortunately this is a new computer and I still have the WSL on my old workstation. Going to restore to a different dist name. "Ubuntu" is probably a bit too generic.

zachsa commented 3 years ago

I frequently have files disappear. It's very easy to tell since I'm working in a repository and all of a sudden I get a long list of 'deleted' files. Also, the repository frequently corrupts and has to be reinstalled

(frequently = a couple times per year)

Pramuspl commented 3 years ago

I'm using WSL2 with Docker Desktop. This morning I experience Docker Desktop 'failing to initialize'. After investigation it turned out that the settings.json is filled with NULLs. After removing it, I managed to run it but the Ubuntu distro was gone. I opened MS Store and Launched (not reinstalled, just Launched!) Ubuntu and ended up with a fresh instance and all my files gone.

I understand that there might be bugs but how come is it allowed for MS Store/Ubuntu installer to overrride such sensitive data with no permission from the user whatsoever? This is unacceptable and I became yet another person who completely lost trust in WSL overnight.

mateusleon commented 3 years ago

With me the problem started this morning when I turned on my PC and opened VS Code. WSL2 was somewhat working but, when I tried to run a server and open the URL through my browser on Windows, I was receiving ERR_CONNECTION_REFUSED.

So I've checked some configurations, firewall, and on my antivirus (Kaspersky Free), on the Settings, the protection through virtualization was active but there was a warning stating that neither the hardware nor the software were able to handle virtualization.

Then I proceeded to UEFI and checked if the SVM was enabled. For a test, I've disabled the option, restarted the PC, proceeded to desktop, shut down and reenabled SVM. But then the problem increased because now I wasn't even able to access WSL2, also none of the CMD commands were working.

I was then unable to open Ubuntu through MS Terminal or VS Code. VS Code was stating that "No distros were found", but when I looked on MS Store Ubuntu status was marked as Installed.

I was worried that I could lost all my settings and files but I've proceeded disabling / reenabling Linux Subsystem for Windows feature and it worked!

I cannot give you a more in depth analysis because I'm not very experienced using WSL2 but, somehow, something messed up with the configurations of it and by toggling the feature off / on fixed the issue.

evantkchong commented 2 years ago

Encountered some variant of this today. Opened VSCode workspace and an error message came up saying that the project directory no longer exists. Found that my ubuntu user home directory was gone. In its place was a directory for wslg. A cursory Google search indicated that some people were able to get back their accounts and files by restarting. Worked.

bitnahian commented 2 years ago

Incredible. I've been facing the same issues. Every time I restart my PC, my WSL distro installation magically disappears.

KamenRiderKuuga commented 2 years ago

Just to provide reference for others, in my case, the same problem occurs when I create content in the /dev directory, but for other directories, no such thing happened.

JVKeller commented 2 years ago

I just restored a backup I took with acronis and WSL2 is just gone... I reeeaaaaaly needed the files to be there so I could renew some SSL certificates easily.

daniloromero commented 2 years ago

It happened to me last week, I was setting an environment to test a hugging face model, installed everything. but it didn't run for some error. it was late so the next day I open the terminal again and repository, environment and libraries gone. Windows build 19042.1415 Why is this issue closed as it keeps happening

ferrlen commented 2 years ago

Wow, also happened to me. I thought I was doing something wrong, but every time I closed the terminal, my installations would be gone. I need to reinstall npm pretty much every time. When will this be finally fixed? Again, why is this issue closed if it's still happening and no solution has been provided?

dn-scribe commented 2 years ago

Same problem. Installed golang, and even when openning another terminal - it is not available. Configuration: Win10+WSL+Ubuntu18.04

Meanwhile moving to native linux

dambem commented 2 years ago

This just occurred to me this morning, incredibly frustrating as I had some critical libraries on WSL that I was working on. Moving back to dual-boot after this as this is just crazy. Windows 11 Pro, build 22581.200.

Planning on moving directly to Mac after this, as if you can't trust an OS not to get itself deleted, can't really trust it to do anything else.

engina commented 2 years ago

This just occurred to me this morning, incredibly frustrating as I had some critical libraries on WSL that I was working on. Moving back to dual-boot after this as this is just crazy. Windows 11 Pro, build 22581.200.

Planning on moving directly to Mac after this, as if you can't trust an OS not to get itself deleted, can't really trust it to do anything else.

Too dramatic you might sound to some but this is precisely what happened to me as well. I was trying to go back to windows from mac. And this bug just sent me back to mac 2y ago.

T w o y e a r s a g o.

spanza929 commented 2 years ago

Just happened to me this morning. I am running Microsoft Windows [Versión 10.0.22593.1] Edición Windows 11 Pro Lots of adjectives and adverbs come to mind, but I will stay with unbelievable. It's now starting its third year since first reported by @engina.

codebruch commented 2 years ago

Same here again, thats the end of WSL for me. Unbelivable how can an entire OS wipe out itself???

dgreenwald-ccs commented 2 years ago

Same just happened to me! My default Ubuntu distro gone, poof! This is very bad. C:\Users...\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc is empty. Interestingly, I had another distro that I downloaded directly (Amazon linux) and its still there just fine. Maybe that's a workaround download the distro instead of using windows store...

augustl commented 2 years ago

(where's the delete button when you need one?)

aaronaan commented 2 years ago

still happens. opensuse-42 install gone

TW4177 commented 2 years ago

Add me to this. Migrating my machine from a workgroup to a company domain wiped out my VM+dev. environment and replaced it with an OOB Canonical VM image. It looks like wsl --unregister got run as a "side effect". Searching my C: for leftover deleted blocks from the VHDX or to reconnect the file itself has been fruitless.

wsl --unregister is extremely dangerous! It should not run without interactive confirmation or an extra flag to confirm. Preferably, it should archive away the unregistered data, and another option used in a separate execution should do the final disposal.

@bpulliam's deflection above is shameful and does nothing to help customers with this problem. This ticket and all related should remain open and escalated until this safety defect is remedied. It clearly has already cost Microsoft customers 1000s of hours of work.

dgreenwald-ccs commented 2 years ago

Going forward I only use distros that I download to a specific folder that I control. I don't touch the default ubuntu or any windows store distros. A bit more work but so far I have not lost anything this way...

TW4177 commented 2 years ago

@dgreenwald-ccs I'm not 100% clear if this is an inherent WSL risk (i.e., --unregister, so any VM instance is vulnerable), or a side effect of the Canonical distro's installation tools. I'm considering a vanilla Hyper-V VM and installing Ubuntu manually. Less "integrated", but perhaps safer.

dgreenwald-ccs commented 2 years ago

-unregister will not delete a downloaded distro. The .exe & tar.gz files will remain. If you run it again it will simply re-register. I just re-tested this because you made me have doubts..

TW4177 commented 2 years ago

The issue goes more to the VM image that contains one's data; all refs I've read on --unregister say it removes it.

dgreenwald-ccs commented 2 years ago

Yes we are on the same page about the VM image. However I can confirm that --unregister only deletes images that it manages, for example the default Ubuntu image or anything from the Windows Store. If you have a 'custom' OS image that you either build or download from a different source, windows will not go out and delete the image because it is managed by you.. This is how you can create a WSL image: https://github.com/Microsoft/WSL-DistroLauncher Or for example I am using this bre-built Amazon Linux 2 image: https://github.com/yosukes-dev/AmazonWSL/

TW4177 commented 2 years ago

Beg to differ on the destructiveness of --unregister:

PS C:\Users\tw4177> cd AppData\Local\Amazon2WSL
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> dir

    Directory: C:\Users\tw4177\AppData\Local\Amazon2WSL

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         7/29/2022   2:48 PM                assets
-a----         7/29/2022   2:48 PM        2768384 Amazon2.exe
-a----         7/29/2022   2:48 PM       86325141 rootfs.tar.gz

PS C:\Users\tw4177\AppData\Local\Amazon2WSL> .\Amazon2
Using: C:\Users\tw4177\AppData\Local\Amazon2WSL\rootfs.tar.gz
Installing...
Installation complete
Press enter to continue...
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> dir

    Directory: C:\Users\tw4177\AppData\Local\Amazon2WSL

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         7/29/2022   2:48 PM                assets
-a----         7/29/2022   2:48 PM        2768384 Amazon2.exe
-a----         7/29/2022   2:49 PM      334495744 ext4.vhdx
-a----         7/29/2022   2:48 PM       86325141 rootfs.tar.gz

PS C:\Users\tw4177\AppData\Local\Amazon2WSL> wsl -l -v
  NAME                    STATE           VERSION
  rancher-desktop         Stopped         2
  Amazon2                 Stopped         2
  rancher-desktop-data    Stopped         2
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> wsl -d Amazon2
-bash-4.2#
... poke around ...
-bash-4.2# exit
logout
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> wsl -l -v
  NAME                    STATE           VERSION
  rancher-desktop         Stopped         2
  Amazon2                 Running         2
  rancher-desktop-data    Stopped         2
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> wsl --terminate Amazon2
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> wsl -l -v
  NAME                    STATE           VERSION
  rancher-desktop         Stopped         2
  Amazon2                 Stopped         2
  rancher-desktop-data    Stopped         2
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> wsl --unregister Amazon2
Unregistering...
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> wsl -l -v
  NAME                    STATE           VERSION
  rancher-desktop         Stopped         2
  rancher-desktop-data    Stopped         2
PS C:\Users\tw4177\AppData\Local\Amazon2WSL> dir

    Directory: C:\Users\tw4177\AppData\Local\Amazon2WSL

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         7/29/2022   2:48 PM                assets
-a----         7/29/2022   2:48 PM        2768384 Amazon2.exe
-a----         7/29/2022   2:48 PM       86325141 rootfs.tar.gz

PS C:\Users\tw4177\AppData\Local\Amazon2WSL>

The VM image file is GONE!

dgreenwald-ccs commented 2 years ago

Hmm I see you are correct. Thank you for pointing that out. I will now regularly backup the vhdx file. Very bad.

dgreenwald-ccs commented 2 years ago

@TW4177 I believe this tool would be helpful here, not quite sure how yet. Will need to invest some time to figure out a workflow: https://github.com/DDoSolitary/LxRunOffline

ikalkov commented 2 years ago

Happened to me today on Windows 11 (10.0.22000)

@bpulliam, seems like it wasn't quite fixed. Could you please point to a related open issue or reopen this one?

ihardyslide commented 2 years ago

Happened today to me the second time already. Seems like some stuff was left original (because timestamps in %appdata% say so), but my home dir was brutally cleared. I have lost a good bit of work.

DmitriyMolch commented 1 year ago

me too

jmhill commented 1 year ago

Encountered some variant of this today. Opened VSCode workspace and an error message came up saying that the project directory no longer exists. Found that my ubuntu user home directory was gone. In its place was a directory for wslg. A cursory Google search indicated that some people were able to get back their accounts and files by restarting. Worked.

This just happened to me also. I was in the middle of working (I use vscode devcontainers for all of my work, with repos cloned into the WSL2 file system, use windows terminal) and took a break so allowed WIndows 11 to restart to apply an update. When I came back to my restarted system, my regular user home directory was gone and only the "wslg" user's home directory (which I had never seen before, so not sure where that came from) was present.

As stated, simply restarting seems to have worked to restore everything to working order. No idea what's happening but if I can be of service to provide more information for debugging purposes please let me know. I love using WSL but this rattled me a bit.

theonlyfoxy commented 1 year ago

Encountered some variant of this today. Opened VSCode workspace and an error message came up saying that the project directory no longer exists. Found that my ubuntu user home directory was gone. In its place was a directory for wslg. A cursory Google search indicated that some people were able to get back their accounts and files by restarting. Worked.

This just happened to me also. I was in the middle of working (I use vscode devcontainers for all of my work, with repos cloned into the WSL2 file system, use windows terminal) and took a break so allowed WIndows 11 to restart to apply an update. When I came back to my restarted system, my regular user home directory was gone and only the "wslg" user's home directory (which I had never seen before, so not sure where that came from) was present.

As stated, simply restarting seems to have worked to restore everything to working order. No idea what's happening but if I can be of service to provide more information for debugging purposes please let me know. I love using WSL but this rattled me a bit.

Same Issue happened to me and restarting solved it. You have to do wsl --shutdown to resolve it.

KonanTheLibrarian commented 1 year ago

NOT FIXED! See https://github.com/microsoft/WSL/issues/9728 ghost reappears 3 years later. This is serious, please backup WSL $HOME area to Windows as follows: wsl you $ cd ~/.. wsl you $ sudo tar --numeric-owner --one-file-system -p -cJf /mnt/c/Users/$USER/wslbackup${USER}9.tar.xz ./$USER NB. Change the file name each time. Note this assumes the Windows user is the same as the WSL user! Adjust accordingly.

The root causes of this bug (Massive Memory Corruption) continues to haunt WSL2 users a whole 3 years later with strange sightings of vanishing ALL $HOME work, developers who failed to backup howling in pain, also lost WSL2 DNS network connections. Perhaps WSL2 developers need to place a MEMORY TOP guard marker, and monitor it for being overwritten. Just thoughts.

ashleyghooper commented 1 year ago

Add me to this. Migrating my machine from a workgroup to a company domain wiped out my VM+dev. environment and replaced it with an OOB Canonical VM image.

Ah, this sounds somewhat similar to the suspected cause of my disk image getting blown away yesterday. We were migrating from logging in with the old style DOMAIN\username to using email-like UPNs. I've never thought it would make any difference which of the two were used, but possibly there's an edge case here with WSL.

dgreenwald-ccs commented 1 year ago

Our team now uses WSL like vagrant, as a disposable linux instance. Everything is built out of a single bash script. As long as code changes were pushed nothing, will be lost. I'm considering looking into a Samba share from windows to linux instead of storing code on the wsl instance. (/mnt/c/... has issues with change notifications in vscode).

lschweiss commented 1 year ago

Hit this bug today.

I see Microsoft has closed every issue related to it. That's it I have no reason ever to run Microsoft again.

denisvmedia commented 1 year ago

2 years ago I wrote this comment: https://github.com/microsoft/WSL/issues/4739#issuecomment-785814294 And now I see this bug (which is closed, although still being actual). I see how I was right in my fears of losing the data. In fact, I'm so happy I never switched to WSL2. WSL1 works just perfectly for me.

ashleyghooper commented 1 year ago

My colleague lost his entire WSL2 .vhdx file by the same mechanism as me - that is immediately after switching from logging in to Windows using the username format DOMAIN\username to using first.last@domain.com.