microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.48k stars 822 forks source link

WSL 2 needs Hyper-V enabled. But this conflicts with 3rd party software ( No nested VT-x in the root domain ) #5030

Closed kwinz closed 4 years ago

kwinz commented 4 years ago

WSL 2 needs Hyper-V enabled. But this conflicts with 3rd party software like VMware or Virtualbox.

It is the same issue as why those tools will not run with Windows Defender Credential Guard enabled.

The official solution as per the FAQ is that VMware and Virtualbox change their software so it uses the Hyper-V backend instead of their own via the Hypervisor Platform API. [2] However this API has been buggy for the last few Windows builds and after 2 years it is still not stable. [4]

Wouldn't it be easier to just enable ExposeVirtualizationExtensions globally? Set-VMProcessor -VMName <VMName> -ExposeVirtualizationExtensions $true So that if Vmware or Virtualbox are started they will be able to access the Intel VT-x (and AMD equivalent) instructions?

Basically I finally want to be able to run WSL2 while still keeping the Virtualbox support that I rely on. And I don't want to wait another 2-3 years until the Hypervisor Platform API matures and then probably still having performance regressions.

Can I enable ExposeVirtualizationExtensions for the client that the main Windows 10 OS runs in. Not talking about inside WSL2 client here. I don't know what to put for the VMName though. Is this feasible?

[1] https://superuser.com/questions/1208850/why-cant-virtualbox-or-vmware-run-with-hyper-v-enabled-on-windows-10/1291879#comment2284544_1291879 [2] https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq [3] https://docs.microsoft.com/en-us/virtualization/api/ [4] https://forums.virtualbox.org/viewtopic.php?t=90853

zippaaa commented 4 years ago

WSL 2 needs Hyper-V enabled. But this conflicts with 3rd party software like VMware or Virtualbox.

Virtualbox 6.1 works with WSL 2

изображение

kgraefe commented 4 years ago

Does it work well nowadays? When I tried that a few months ago performance within VBox was horrible. :-(

kwinz commented 4 years ago

According to a Virtualbox dev/employee (posted yesterday):

Guess why I wrote "Of course we tried to convince Microsoft that providing nested VT-x would be the easiest solution for everyone, but they completely refused to consider this direction. They are pushing everyone to use the WHv interface which VirtualBox is using to talk to Hyper-V.

https://forums.virtualbox.org/viewtopic.php?f=6&t=90853&start=240#p474299

I am not saying stop developing the Hypervisor Platform API. This issue is about also exposing nested virtualization additionally.

hez2010 commented 4 years ago

+1 for make ExposeVirtualizationExtensions enable globally and enable nested virtualization for 3-rd party virtualization softwares.

kgraefe commented 4 years ago

So is this resolved?

kwinz commented 4 years ago

@therealkenc genius. Closing the issue without any comment or reply. Please forward the issue to the proper people if you are not the right person to tackle this. Thanks!

therealkenc commented 4 years ago

However this API has been buggy for the last few Windows builds and after 2 years it is still not stable. [4]

For stability issues related to Vbox, feel encouraged to continue the discussion at the link you provided. Third party VM vendors are working closely with MSFT on these issues; they are being taken seriously.

There is an issue for nested virtualization scenarios over at #4193, which for the time being remains open.

For questions/suggestion about the hypervisor platform, see this user voice.

For broad discussion and commentary, reddit and twitter. Not because no one cares. Because they are better platforms for general discussion and commentary.

kwinz commented 4 years ago

Thanks for the reply @therealkenc. #4193 is already linked above, but a different problem.

I am happy that MSFT is supporting its partners use of Hypervisor Platform API. But again different issue from nested virtualization in root domain.

I will check if there is a matching issue on user voice when I have time. (Other's contributions welcome).

kwinz commented 4 years ago

This is the closest user voice issue that I found: https://windowsserver.uservoice.com/forums/295050-virtualization/suggestions/39471682-allow-host-to-run-other-hypervisor-via-nested-vir

sscdarnold commented 4 years ago

As of VMware's Workstation version 15.5.5, Windows Hyper-V and VMware Workstation (again, 15.5 and greater) can coexist.

Here's VMware's take on that: https://blogs.vmware.com/workstation/2020/05/vmware-workstation-now-supports-hyper-v-mode.html

uschindler commented 4 years ago

What's going on with VirtualBOX? Any news on this front?

AlexeiScherbakov commented 4 years ago

As of VMware's Workstation version 15.5.5, Windows Hyper-V and VMware Workstation (again, 15.5 and greater) can coexist.

Here's VMware's take on that: https://blogs.vmware.com/workstation/2020/05/vmware-workstation-now-supports-hyper-v-mode.html

They can coexist but 3D acceleration is NOT working, because nested virtualization is not working :-(

ewanm89 commented 4 years ago

As of VMware's Workstation version 15.5.5, Windows Hyper-V and VMware Workstation (again, 15.5 and greater) can coexist.

Here's VMware's take on that: https://blogs.vmware.com/workstation/2020/05/vmware-workstation-now-supports-hyper-v-mode.html

This is not nested virtualization, but the use of hyper-v interface. This is more like vmware panel controlling hyper-v, and is precisely what is not wanted.

Weirdo1312 commented 4 years ago

it also conflicts with tencent gameloop emulator

policeman0077 commented 4 years ago

the latest VirtualBox6.1 using hyper-v backend is still very choppy and unstable

onomatopellan commented 4 years ago

No wonder Virtualbox has always been unstable with Hyper-V backend. Since May 2018 the implementation was broken and CPU registers under HyperV had wrong values. That means instability, crashes, data corruption in downloads and disk, and so on.

This fix in latest test builds (>=r140351) should make Virtualbox finally usable with the Hyper-V backend.

geoffrey-vl commented 4 years ago

No wonder Virtualbox has always been unstable with Hyper-V backend. Since May 2018 the implementation was broken and CPU registers under HyperV had wrong values. That means instability, crashes, data corruption in downloads and disk, and so on.

This fix in latest test builds (>=r140351) should make Virtualbox finally usable with the Hyper-V backend.

Note that this patch is not yet included in the September '20 release of VirtualBox, i.e. 6.1.14. I've tried a development release (download can be found here) which currently installs version 140491(aka 6.1.97) I for now I found the stability and performance a whole lot better than the "stable" 6.1.14 release. It's definitely worth the try. I'm running a 64-bit Win10 Pro (1909) host with Intel CPU and Hyper-V enabled (for Docker). Inside my VM I'm running Ubuntu 18.04 LTS, KVM virtualisation, VMSVGA GPU and 3D enabled. Benchmarks:

=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     VMware, Inc.
    GL_RENDERER:   SVGA3D; build: RELEASE;  LLVM;
    GL_VERSION:    2.1 Mesa 20.0.8
=======================================================
[build] use-vbo=false: FPS: 230 FrameTime: 4.348 ms
[build] use-vbo=true: FPS: 546 FrameTime: 1.832 ms
[texture] texture-filter=nearest: FPS: 237 FrameTime: 4.219 ms
[texture] texture-filter=linear: FPS: 252 FrameTime: 3.968 ms
[texture] texture-filter=mipmap: FPS: 255 FrameTime: 3.922 ms
[shading] shading=gouraud: FPS: 272 FrameTime: 3.676 ms
[shading] shading=blinn-phong-inf: FPS: 156 FrameTime: 6.410 ms
[shading] shading=phong: FPS: 323 FrameTime: 3.096 ms
[shading] shading=cel: FPS: 253 FrameTime: 3.953 ms
[bump] bump-render=high-poly: FPS: 285 FrameTime: 3.509 ms
[bump] bump-render=normals: FPS: 248 FrameTime: 4.032 ms
[bump] bump-render=height: FPS: 231 FrameTime: 4.329 ms
[effect2d] kernel=0,1,0;1,-4,1;0,1,0;: FPS: 378 FrameTime: 2.646 ms
[effect2d] kernel=1,1,1,1,1;1,1,1,1,1;1,1,1,1,1;: FPS: 256 FrameTime: 3.906 ms
[pulsar] light=false:quads=5:texture=false: FPS: 98 FrameTime: 10.204 ms
[desktop] blur-radius=5:effect=blur:passes=1:separable=true:windows=4: FPS: 62 FrameTime: 16.129 ms
[desktop] effect=shadow:windows=4: FPS: 168 FrameTime: 5.952 ms
[buffer] columns=200:interleave=false:update-dispersion=0.9:update-fraction=0.5:update-method=map: FPS: 49 FrameTime: 20.408 ms
[buffer] columns=200:interleave=false:update-dispersion=0.9:update-fraction=0.5:update-method=subdata: FPS: 64 FrameTime: 15.625 ms
[buffer] columns=200:interleave=true:update-dispersion=0.9:update-fraction=0.5:update-method=map: FPS: 66 FrameTime: 15.152 ms
[ideas] speed=duration: FPS: 235 FrameTime: 4.255 ms
[jellyfish] <default>: FPS: 411 FrameTime: 2.433 ms
Error: SceneTerrain requires Vertex Texture Fetch support, but GL_MAX_VERTEX_TEXTURE_IMAGE_UNITS is 0
[terrain] <default>: Unsupported
[shadow] <default>: FPS: 178 FrameTime: 5.618 ms
[refract] <default>: FPS: 180 FrameTime: 5.556 ms
[conditionals] fragment-steps=0:vertex-steps=0: FPS: 385 FrameTime: 2.597 ms
[conditionals] fragment-steps=5:vertex-steps=0: FPS: 253 FrameTime: 3.953 ms
[conditionals] fragment-steps=0:vertex-steps=5: FPS: 255 FrameTime: 3.922 ms
[function] fragment-complexity=low:fragment-steps=5: FPS: 255 FrameTime: 3.922 ms
[function] fragment-complexity=medium:fragment-steps=5: FPS: 255 FrameTime: 3.922 ms
[loop] fragment-loop=false:fragment-steps=5:vertex-steps=5: FPS: 255 FrameTime: 3.922 ms
[loop] fragment-steps=5:fragment-uniform=false:vertex-steps=5: FPS: 255 FrameTime: 3.922 ms
[loop] fragment-steps=5:fragment-uniform=true:vertex-steps=5: FPS: 254 FrameTime: 3.937 ms
=======================================================
                                  glmark2 Score: 237 
=======================================================
kstoykov commented 4 years ago

Everywhere is written that Docker needs Hyper-V to be enabled because of WLS 2. Of course, after enabling Hyper-V, the VirtualBox can't start any windows client VM (Linux client VM are working horribly). Anyway I thought that I could disable/enable Hyper-V when needed and hence switching between docker and Vritual Box (depent on needs).

Suddenly I figured out that Docker is working (at least the "getting started" example) with DISABLED Hyper-V and Containers. Sadly Virtual Box still can't start any VM.

Any thoughts?

onomatopellan commented 4 years ago

@kstoykov That means you are probably under the hypervisor. Not only hyper-v or containers but Sandbox, VirtualizedBasedSecurity and more can also enable it... Open elevated PowerShell and run bcdedit.exe. If you see hypervisorlaunchtype Auto then Virtual Box 6.1.14 won't start. You need to either disable it, wait for an update or install the test builds.

Spongman commented 4 years ago

can't believe this is still broken. wsl2 is dead to me until this is fixed :(

ivanzud commented 4 years ago

I just updated to 20251.1 and WSL2 and VMware Workstation 16 works together. hypervisorlaunchtype is set to Auto. The issue is finally fixed for me.

kwinz commented 3 years ago

@ivanzud Using proper Nested Virtualization ( ExposeVirtualizationExtensions in the root domain ) or with the Hypervisor Platform API (WHVP) workaround?

AlexeiScherbakov commented 3 years ago

VMWare Player 16 works for me with WSL 2, but nested virtualization is not supported and graphics acceleration too :-(

VMWare version - 16.0.0 build-16894299 Windows 10 version - 19041

Virtualize Intel VT-x cannot be set -> graphics acceleration cannot be enabled I use VMWare player only for old games and with WSL 2 VMWare is useless for this.

kstoykov commented 3 years ago

@AlexeiScherbakov I checked WebGL in a browser and it is working pretty well. Of course, not as on the host, but I think it is not running on CPU

pcmaniac23 commented 3 years ago

VMWare Player 16 works for me with WSL 2, but nested virtualization is not supported and graphics acceleration too :-(

VMWare version - 16.0.0 build-16894299 Windows 10 version - 19041

Virtualize Intel VT-x cannot be set -> graphics acceleration cannot be enabled I use VMWare player only for old games and with WSL 2 VMWare is useless for this.

This is precisely my problem. I want wsl2 alongside 3rd Party Virtualization solutions WITH 3D graphics acceleration enabled in order to keep playing old games. Until this problem is resolved, I am not going to update from wsl to wsl2.

jonilearncode commented 3 years ago

Still not working. Bummer!!

It really sucks need to manually 'switch' this hyper off/auto if I want to get VBOX(W10) working or WSL2...

hypervisorlaunchtype auto --> enables WSL2 & disables w10 VBOX hypervisorlaunchtype off --> disables WSL2 & enable w10 VBOX

charlesmorin1 commented 2 years ago

You can also simply use Windows Features and enable/disable WIndows Virtual Machine. Both options; bcdedit and Win Features require a reboot,

Ridiculous in the post-2020 Cloud era we have to deal with these incompatibilties and lack of vendor support on these issues.

ChrisKader commented 2 years ago

April 24th, 2022 and we still need to toggle hypervisorlaunchtype and reboot in order to switch between using WSL2 or VirtualBox.

crutch12 commented 2 years ago

Bump

masoudei commented 2 years ago

Still nested virtualization on VMWare not working within "bcdedit /set hypervisorlaunchtype auto" mode, So we can not have Docker Desktop ( Hyper-V or WSL/2) in Windows & VMWare with nested virtualization on windows at the same time! Shhhhhhhh

feffi commented 2 years ago

this is so ridiculous

eric-schleicher commented 1 year ago

Wow so shameful.

eric-schleicher commented 3 months ago

Sorry, it's not in microsoft's interest to allow this, so they've intentionally done this to make that not possible.