Store WSL isn't accessible from Session 0

OneBlue commented 1 year ago

Version

Multiple versions are affected

WSL Version

[x] WSL 2
[x] WSL 1

Kernel Version

No response

Distro Version

No response

Other Software

No response

Repro Steps

Store WSL currently isn't accessible from session 0 contexts. This means that it's not possible to interact with WSL from remote sessions such as ssh or psremote.

If wsl.exe is called from session 0, it will exit and display:

Access is denied.

on stdout.

yanorei32 commented 1 year ago

I think this issue is already tracked internally. However, this problem continues even now.

Release Notes for Windows Subsystem for Linux in the Microsoft Store

...

Known Issues:

Launching Windows Subsystem for Linux from session zero does not currently work (for example from an ssh connection).

https://learn.microsoft.com/en-us/windows/wsl/store-release-notes#known-issues

benhillis commented 1 year ago

Known issue, we're working on a solution.

bagong commented 1 year ago

This is a deal breaker for some. Would be nice to get notice when it is solved. Thanks!

vakata commented 1 year ago

Please fix this - I cannot remotely login to WSL using the builtin OpenSSH server, my scheduled tasks also stopped working.

brentmjohnson commented 1 year ago

Impact of this issue is complicated by the fact the store packaged version appears to now be required on the latest windows 11 insiders build (10.0.25267). Still looking for workarounds...

https://github.com/microsoft/WSL/issues/9355 https://github.com/microsoft/WSL/issues/9373

enachi commented 1 year ago

Please fix this - I cannot remotely login to WSL using the builtin OpenSSH server, my scheduled tasks also stopped working.

I had to uninstall store version of WSL to be able to run ssh server at system startup again

bagong commented 1 year ago

If policies make it difficult to enable Windows openssh to run WSL, an alternative/workaround might be to run a second openssh server (using a different port) within WSL. At the moment this seems not to be possible. But maybe that possibility can be enabled?

rfc2119 commented 1 year ago

I am surprised by this. By default, I can access wsl from the native OpenSSH server by setting HKEY_LOCAL_MACHINE\SOFTWARE\OpenSSH to C:\Windows\System32\wsl.exe. I am using the store version with Ubuntu LTS 22.04

My problem is actually this: every time I initiate a SSH connection, I get a new WSL shell. I am unable to return back to the same session in case of disconnection events; all my background processes are gone. Each connection is therefore a new WSL instance.

Should I open another ticket for this ?

bagong commented 1 year ago

I am surprised by this. By default, I can access wsl from the native OpenSSH server by setting HKEY_LOCAL_MACHINE\SOFTWARE\OpenSSH to C:\Windows\System32\wsl.exe. I am using the store version with Ubuntu LTS 22.04

I've tried it back and forth on different installs several times, and it never worked. It's a few months ago - should it be fixed and noone here knows? Many have the issue, and it's even acknnowledged by MS... Weird!

My problem is actually this: every time I initiate a SSH connection, I get a new WSL shell. I am unable to return back to the same session in case of disconnection events; all my background processes are gone. Each connection is therefore a new WSL instance.

Should I open another ticket for this ?

I don't think so: interrupted ssh sessions are like logoffs, like closing a terminal emulator (historically speaking closing a terminal emulator is like logging off ;-) ) - you wouldn't expect to get your jobs back there, right... You want to look into tmux or something alike. They keep processes running wile you're off, and you can attach back to them from another shell...

rfc2119 commented 1 year ago

I've tried it back and forth on different installs several times, and it never worked. It's a few months ago - should it be fixed and noone here knows? Many have the issue, and it's even acknnowledged by MS... Weird!

This looks weird indeed. Let me know if any of you would like more details. I just installed the "Ubuntu" app in the store using a local account.

You want to look into tmux or something alike. They keep processes running wile you're off, and you can attach back to them from another shell

It's exactly that! I don't find my tmux sessions when I log back in. In fact, I don't find the tmux process at all!

bagong commented 1 year ago

It's exactly that! I don't find my tmux sessions when I log back in. In fact, I don't find the tmux process at all!

I have no experience with tmux in WSL, so I can't comment from experience. But I'd think about whether your WSL - instance keeps running all the time, or wheter it has had a "reboot" in your time off - even tmux doesn't survive that ;-) Though maybe it can be made to autostart and continue processes? Dunno. I know that WSL's wakeup behavior from suspend is broken (it was in my version) - maybe another reason why tmux might vanish?

Or maybe you need to start your tmux processes in a local shell and let that run? Maybe you can then attach to running tmux from ssh and not shut WSL down by logging off?

And just to let others know: I did install the MS Store update again, and was immediately cut out of ssh-ing in. Fortunately it is enough to uninstall via the App-Icon that appears after the update, and the previous state returns... (i.e. no total reinstall of WSL required)

And yes, if I started tmux in a local shell and kept it running, I can ssh in and out multiple times and see that tmux live on happily...

Final remark for others like me who didn't realize (what for again others is likely self-evident): if it's just about ssh-ing into a running WSL from the outside (i.e. not about actually "booting" wsl): that is possible with the store version of wsl too. You access the host into a shell that doesn't depend on wsl, and from there you ssh into the running wsl: ssh -p n localhost. I guess sshd inside wsl needs to listen on a different port (n) than "outer"-Window's sshd (edit /etc/ssh/sshd_config). For my requirements that's actually tolerable ;-) , and one might consider the inability to start/stop wsl in a ssh-schell a protection of running processes inside wsl.

joes commented 1 year ago

As @bagong mentioned it is possible to install and run a second OpenSSH server (using a different port) in your WSL instance as a workaround.

This can then be combined with using the ProxyJump directive to transparently connect to the second OpenSSH server by jumping through the win32 OpenSSH server. The directive can be added to the remote system's ~/.ssh/config:

Host openssh_win32
    Hostname your_computer.local
    User windows_username

Host openssh_wsl
   ProxyJump openssh_win32
   User wsl_username
   HostName localhost
   Port 2222

The above example assumes that the second OpenSSH server in wsl is running on port 2222.

Using this configuration it is then possible to connect to the wsl instance via the hostname openssh_wsl:

ssh openssh_wsl

ParagDoke commented 1 year ago

While it is possible to get inside the WSL instance based on the proxy jump idea from @joes, this issue still needs to be fixed for use cases such as attempts to install a new WSL distribution on a remote Windows system over SSH.

In other words anything involving the WSL executable besides getting into an instance.

yo1dog commented 1 year ago

My use case is accessing WSL remotely after a cold boot.

I can turn on the PC remotely via WOL and SSH into a win32 OpenSSH server, but cannot start WSL from said SSH connection (session 0). Nor is it possible to start WSL automatically on boot before a user logs in to Windows (also session 0?). For example, you cannot use Task Scheduler's "Run whether user is logged on or not" option to start wsl.exe.

My workaround has been to use RDP just to login remotely and start WSL (which is started automatically on user login by Task Scheduler). Then I can SSH directly into WSL.

Does anyone know of a solution for either remotely initiating a Windows user login or remotely obtaining session 1 without the GUI overhead of the RDP/Remote Desktop app? Ideally command-line only.

ParagDoke commented 1 year ago

@yo1dog In an entirely different context, I know of a solution (batch file) that runs from the command line with GUI overhead, but:

No certificate prompt on RDP client (no blanket suppression, suppress only for the IP / FQDN of your server)
No wait on prompt for things like privacy settings, etc ... some 6 radio buttons that appear on an OOB Windows deployment

This batch file uses openssl, sed to obtain, mangle and register the cert hash for a given server to avoid that cert warning.

On the server, I place a different batch file in the Startup folder (your idea uses Task Scheduler). This could use the timeout command to sleep for a bit, and then logoff from the server (to gracefully terminate the RDP client) after your WSL instance has started.

%1 is the IP / FQDN of your server.

openssl s_client -connect %1:3389 < NUL 2>NUL | openssl x509 -fingerprint -noout | sed -e 's/://g' > %TEMP%\%1-rdp-fingerprint.txt
for /f "tokens=2 delims==" %%a in ('type %TEMP%\%1-rdp-fingerprint.txt') do (
    reg add "HKCU\Software\Microsoft\Terminal Server Client\Servers\%1" /v CertHash /t REG_BINARY /d %%a /f
)
start mstsc /f /v %1

To avoid OOB prompts, this needs to run even before the logon:

reg add "HKLM\Software\Policies\Microsoft\Windows\OOBE" /v DisablePrivacyExperience /d 1

For the script to be run at logon, I believe you already have one.

cmdkey /generic:TERMSRV/<IP or FQDN> /user:<username> /pass:<passwd> will let you save the password for logging in using an RDP client automatically.

gmax9803 commented 1 year ago

@benhillis Has the team made any progress on this issue? And if not is there any quick fix to make this work?

arrmo commented 1 year ago

Also curious here - this is a bit painful, can't execute a WSL (bash) script from a service (Session ID 0).

Thanks!

aki-k commented 1 year ago

@arrmo I was able to resolve that problem by doing this:

https://github.com/microsoft/WSL/issues/8835#issuecomment-1446474057

Edit: I've enabled systemd in WSL by using this:

https://github.com/DamionGans/ubuntu-wsl2-systemd-script

(The enabler script still works, but on newer distros like Ubuntu 22.04 you need a patch from here)

joes commented 1 year ago

@arrmo I was able to resolve that problem by doing this:

#8835 (comment)

Edit: I've enabled systemd in WSL by using this:

https://github.com/DamionGans/ubuntu-wsl2-systemd-script

(The enabler script still works, but on newer distros like Ubuntu 22.04 you need a patch from here)

@aki-k Doesn’t your fix only apply to the non-store version?

aki-k commented 1 year ago

@joes Yes

arrmo commented 1 year ago

@arrmo I was able to resolve that problem by doing this:

Thanks! But ... I admit, not sure about store vs. non-store versions. Can you clarify (and how to check)?

Thanks again.

aki-k commented 1 year ago

Now I'm not so sure any more if my WSL is from the Store:

PS C:\Windows\system32> Get-AppxPackage | findstr /i "linux"
Name              : MicrosoftCorporationII.WindowsSubsystemForLinux
PackageFullName   : MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe
InstallLocation   : C:\Program Files\WindowsApps\MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe
PackageFamilyName : MicrosoftCorporationII.WindowsSubsystemForLinux_8wekyb3d8bbwe

PS C:\Windows\system32> Get-AppxPackage -Name "MicrosoftCorporationII.WindowsSubsystemForLinux"

Name              : MicrosoftCorporationII.WindowsSubsystemForLinux
Publisher         : CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
Architecture      : X64
ResourceId        :
Version           : 1.1.3.0
PackageFullName   : MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe
InstallLocation   : C:\Program Files\WindowsApps\MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe
IsFramework       : False
PackageFamilyName : MicrosoftCorporationII.WindowsSubsystemForLinux_8wekyb3d8bbwe
PublisherId       : 8wekyb3d8bbwe
IsResourcePackage : False
IsBundle          : False
IsDevelopmentMode : False
NonRemovable      : False
IsPartiallyStaged : False
SignatureKind     : Developer
Status            : Ok

PS C:\Windows\system32> dir 'C:\Program Files\WindowsApps\' | findstr /i "linux"
d-----          3/8/2023  12:20 PM                MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_neutral_~_8wekyb3d8bbwe
d-----          3/8/2023  12:20 PM                MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe

Somebody wrote that the Store packages would be installed into C:\Program Files\WindowsApps .

arrmo commented 1 year ago

Interesting - I have no C:\Program Files\WindowsApps\ directory. Thoughts?

Thanks!

aki-k commented 1 year ago

@arrmo It's supposedly hidden (and access rights removed from the user) by default:

https://www.thewindowsclub.com/what-is-windowsapps-folder

Edit: I also can't access it from Windows Explorer but (Run as Administrator) Powershell didn't have any problems :)

arrmo commented 1 year ago

Edit: I also can't access it from Windows Explorer but (Run as Administrator) Powershell didn't have any problems :)

Same here! Thought I was going crazy ... LOL. OK, from Powershell (Admin),

d-----          3/7/2023  12:05 PM                MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_neutral_~_8we
d-----          3/7/2023  12:05 PM                MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d

So are you thinking your fix should work here? Will try it if you think so. Thanks!

aki-k commented 1 year ago

Will try it if you think so.

It was tricky to get started but I described it as well as I could. If you have questions about it, I can help out.

arrmo commented 1 year ago

It was tricky to get started but I described it as well as I could. If you have questions about it, I can help out.

Thanks!! One question ... your script seems to be about systemd, but I'm just trying to launch a bash script - from Session 0. Not sure how / why systemd applies here (but that's likely me!).

Thanks again.

aki-k commented 1 year ago

your script seems to be about systemd

I enable systemd in WSL because I run services in there (sshd, containerd, dockerd) and access the container with a XPRA client (https://xpra.org/).

arrmo commented 1 year ago

Yes, agreed! But I'm just trying to spawn a script, no service inside WSL -> your solution still apply?

Thanks!

selivan commented 1 year ago

I confirm that WSL commands still do not work from scheduled tasks configured (*) Run whether the user is logged on or not with following software versions:

Windows 11 22H2 build 22621.1344
WSL: 1.1.3.0 (full package name MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe) installed with winget

ghost commented 1 year ago

Is there a status change here? Unfortunately, this very much prevents us from automating with Ansible.

nttranbao commented 1 year ago

I am also on the same boat. My goal is to have WSL starts at boot time, without user having to log on.

Workaround that I am using:

Enable auto login. This requires keeping user password in the registry. It's okay for now since I'm running a windows 10 VM on VMWare ESX, and have another script to lock the screen upon user logging in.
Set Windows Terminal to use WSL as default shell, and set to launch on machine start up.

It's not ideal, but at least it works for now, and everything survives after a VM reboot. Tried other methods found on the internet, but either not working anymore, or requires a series of actions.

Hope we'll get a fix soon.

huyz commented 1 year ago

@nttranbao I don't think you need to do #2 "Set Windows Terminal to use WSL as default shell, and set to launch on machine start up".

My workarounds:

Have Task Scheduler start wsl at launch.
Have openssh server start at launch, I configure wsl.conf with either:
```
[boot]
command="service ssh start"
```
or
```
[boot]
systemd=true
```
To ssh to WSL, I make use of @joes's workaround https://github.com/microsoft/WSL/issues/9231#issuecomment-1383674435 by jumping through the Win32 OpenSSH server first.

nttranbao commented 1 year ago

Thanks @huyz for the suggestion. I'm running Windows 10 here so not sure if it makes any difference though.

Btw, for my #2, I was able to switch to Task Scheduler to run wsl in an invisible window, basically to run the following command at log on

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe start C:\Windows\System32\wsl.exe -WindowStyle Hidden

I think it works the way I want, at least for now, until the final fix for this bug is released.

Thanks again,

slacksystem commented 1 year ago

This is frustrating and I'm not sure how to (if even possible) to apply the proposed workarounds to my use case. I don't want to ssh into wsl, I'm sshing into powershell and using that. But sometimes I just want to run a command in wsl from within my powershell session. But it seems I can't do that, even if wsl is running already, if I'm logged in over ssh. Can anyone explain to me how/if I can use the above workingarounds and if not, is there a way I can revert to the non-store wsl?

aki-k commented 1 year ago

is there a way I can revert to the non-store wsl?

You can remove WSL from PC Settings/Apps or with the following Powershell commands:

PS > Get-AppPackage -name "*WindowsSubsystemForLinux*"
or
PS > Get-AppxPackage -name "*WindowsSubsystemForLinux*"

PS > Remove-AppPackage -package MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe -Confirm:$true
or
PS > Remove-AppxPackage -package MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe -Confirm:$true

You can then reinstall the non-store version in admin cmd prompt:

C:\>wsl --install -d Ubuntu-22.04 --web-download

Edit: I just noticed the wsl command options change so you'd need to check if those apply

huyz commented 1 year ago

Does anyone know what we lose by going non-Store? They say the store version will have the latest updates. Does that mean non-Store won't be updated anymore or just that it will lag?

aki-k commented 1 year ago

They say the store version will have the latest updates. Does that mean non-Store won't be updated anymore or just that it will lag?

I don't think there's too much of a difference. v1.2.3 was released yesterday:

https://github.com/microsoft/WSL/releases

slacksystem commented 1 year ago

You can remove WSL from PC Settings/Apps or with the following Powershell commands: @aki-k Will this keep all the files I have on my Ubuntu instance or will I have to transfer them somehow?

aki-k commented 1 year ago

@slacksystem Looks like it's safe to remove the WSL package and install the non-store version of WSL. I just updated this way from v1.1.3 to v1.2.3 (I can't give any guarantees this will work on your system. Copy the ext4.vhdx file to safety first if you're concerned):

C:\>dir C:\users\akik\ext4.vhdx /s

Directory of C:\users\akik\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu22.04LTS_79rhkp1fndgsc\LocalState

17/04/2023 01.23 2 219 835 392 ext4.vhdx

C:\>wsl --shutdown

download the new release from https://github.com/microsoft/WSL/releases

PS > Get-AppxPackage -name "*WindowsSubsystemForLinux*"

PackageFullName : MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe

PS > Remove-AppxPackage -package MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe -Confirm:$true

PS > Add-AppxPackage -path C:\Users\$env:UserName\Downloads\Microsoft.WSL_1.2.3.0_x64_ARM64.msixbundle

C:\>bash

glxgears still works, and "apt-get update; apt-get dist-upgrade" didn't find any packages to update

C:\>wsl -v WSL version: 1.2.3.0 Kernel version: 5.15.90.1 WSLg version: 1.0.51 MSRDC version: 1.2.3770 Direct3D version: 1.608.2-61064218 DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp Windows version: 10.0.19045.2846

slacksystem commented 1 year ago

Thanks for all your help. I'm gonna try this in a little while. Unless anyone pops up with a way that a workaround could apply to my usage that'd let me keep the store version. Hope there's an actual fix for this soon as the loss of functionality on the upgrade is making things more difficult

rfay commented 1 year ago

WSL Maintainers, there's a lot of confusion out there (demonstrated here) about the "store version" vs the "wsl version". Surely they're the same by now?

aki-k commented 1 year ago

about the "store version" vs the "wsl version". Surely they're the same by now?

The downloads at https://github.com/microsoft/WSL/releases are tagged as 'pre-release'. I don't think you would add those to the Windows Store but who knows (MS knows).

bilby91 commented 1 year ago

Downloads from Github didn't work for me. I followed this guide and it worked correctly.

https://learn.microsoft.com/en-us/windows/wsl/install-manual

Was trying to get wsl with ssh session.

aki-k commented 1 year ago

Downloads from Github didn't work for me

You can use the following curl command to download the msixbundle file:

curl -OL https://github.com/microsoft/WSL/releases/download/1.2.3/Microsoft.WSL_1.2.3.0_x64_ARM64.msixbundle

gdragon-git commented 1 year ago

My workarounds:

Have Task Scheduler start wsl at launch.

@huyz How?

I tried "How to Run WSL2 at Startup on Windows" and wsl2host.exe and can't get wsl to work on startup.

Did you get it to work on startup? Can you refer me to the way you did it?

When I startup, I can SSH into my Windows machine but each attempt to run WSL from the Powershell I'm SSH'd to, I get:

"The file cannot be accessed by the system"

How are you able to avoid that without logging in a user ?

huyz commented 1 year ago

@Magnacoder sorry, yeah it doesn't work at startup. I have to turn on auto-login

grant-singleton-nz commented 1 year ago

is there a way I can revert to the non-store wsl?

You can remove WSL from PC Settings/Apps or with the following Powershell commands:
PS > Get-AppPackage -name "*WindowsSubsystemForLinux*"
or
PS > Get-AppxPackage -name "*WindowsSubsystemForLinux*"
PS > Remove-AppPackage -package MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe -Confirm:$true
or
PS > Remove-AppxPackage -package MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.3.0_x64__8wekyb3d8bbwe -Confirm:$true
You can then reinstall the non-store version in admin cmd prompt:
C:\>wsl --install -d Ubuntu-22.04 --web-download
Edit: I just noticed the wsl command options change so you'd need to check if those apply

Trying to follow these instructions - I don't see a --web-download option to wsl --install, is there an alternative method that might work?

I too am completely confused about whether I'm installing the store version or the non-store version, how can I tell after its installed?

For context, I'm trying to configure Windows 2022 as a Jenkins agent. The jenkins agent connects over ssh with a non-admin user and runs some pwsh, cmd and bash commands. Windows is a VM running in a hypervisor that doesn't support nested virtualisation, so I need to use WSL1 (we also get much better performance from WSL1 over WSL2 when running on Windows 10/11 on physical hardware)

grant-singleton-nz commented 1 year ago

Follow up on my previous post. I found that non-store WSL works over SSH if the user is an Administrator but doesn't if they're not. I'd prefer not to give the user full admin rights, does anyone know what the minimum rights that are needed?

aki-k commented 1 year ago

I don't see a --web-download option to wsl --install

Microsoft changes the wsl options from version to version. I have seen this same problem.

I've decided to follow these steps from now on to install the newest WSL release on Windows 10:

download wsl 2 kernel:

https://aka.ms/wsl2kernel
https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi

download wsl 2:

https://github.com/microsoft/WSL/releases

run in admin cmd prompt:

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
dism.exe /online /Add-ProvisionedAppxPackage /packagepath:C:\Users\username\Downloads\Microsoft.WSL_1.3.11.0_x64_ARM64.msixbundle /SkipLicense

restart computer:

shutdown /r /f /t 0

install wsl_update_x64.msi

user cmd prompt:

wsl --set-default-version 2
wsl --install Ubuntu-22.04 --web-download

The last command just installs the WSL Ubuntu-22.04 distro.

microsoft / WSL