search

virtio-win / kvm-guest-drivers-windows

Windows paravirtualized drivers for QEMU\KVM
https://www.linux-kvm.org/page/WindowsGuestDrivers
BSD 3-Clause "New" or "Revised" License
2.04k stars 387 forks source link

virtio-fs service on a win10 QEMU guest fails before restart and user cannot modify created files #550

Closed marcello-pietrobon closed 2 years ago

marcello-pietrobon commented 3 years ago

As probably well known virtio-fs on a Win10(20H2) Windows guest (and maybe other versions too) is unusable for some users.

The way this problem manifests at first is that a user is able to create/delete files or directories on a drive mapped through virtio-fs, but then he cannot modify them in any way (where in the case of a directory this means to add any file to it or remove it).

On a first look this problem appears of the same nature of few others that have been affecting some WinFsp filesystems, but on a further analysis it actually seems to be specific to the virtio-fs driver which, before restart, returns VirtFs->LocalUid == VirtFs->LocalGid == 0 (or better most probably does so).

Once the virtio FS service is restarted (after logging in into the VM at least) the returned uid/gid are correct and the file permissions correctly mapped to those of the underlying linux host filesystem.

A more complete description of this problem is found in virtio-fs with a win10 QEMU guest and WinFsp

I'm available to perform some testing or answer other questions if needed.

Thank you.

Edit: I wonder if the virtio-fs service should be started after some other windows service that I'm not aware of...

daiaji commented 3 years ago

Encountered a similar problem. After logging in to the WIN10 guest, the folder cannot be created in the drive letter mapped by virtio-fs, but everything is normal after restarting the virtio-fs service. Does the virtio-fs service start prematurely?

hammerg commented 3 years ago

This issue should have been resolved in commit 57aedaa3. Are you working with a version that includes this fix?

The virtio-fs service is depending on two other services. The WinFsp.Launcher and the VirtioFsDrv driver. So a manual installation of it should look something like:

C:\> sc create VirtioFsSvc binpath="(your binary location)\virtiofs.exe" start=auto depend="WinFsp.Launcher/VirtioFsDrv" DisplayName="Virtio FS Service"
daiaji commented 3 years ago

Thank you. Does it have a compiled executable?

marcello-pietrobon commented 3 years ago

It works!

Thank you so much for having taken care of this so promptly, very much appreciated.

This is what I did to compile: I went to

marcello-pietrobon commented 3 years ago

There's only one small issues left, I write it here as memo for anyone who may face this problem, but I'm content already with the fix... and for what it concerns me this issue can be closed.

Here's the description of the remaining issue: Everything works, except with some zip files. Some cannot be copied. It's very weird. This is how it goes:

The difference between the two cases seems to me only that in the first case it was me who created the directory.

Edited: I don't know if this is an issue with virtio-fs or with WinFsp. but what I find asthonishing is that if I run fsptool-x64.exe or Icacls on both zip files they give exactly the same result. For example for test1.zip is:

z:>"c:\Program Files (x86)\WinFsp\bin\fsptool-x64.exe" perm z:\test1.zip O:S-1-5-21-2819646085-1004851513-3197712829-1001G:S-1-5-21-2819646085-1004851513-3197712829-513D:P(A;;0x1f01bf;;;S-1-5-21-2819646085-1004851513-3197712829-1001)(A;;0x1201af;;;S-1-5-21-2819646085-1004851513-3197712829-513)(A;;0x1200a9;;;WD) (perm=197609:197121:0775)

z:>Icacls z:\test1.zip z:\test.zip DESKTOP-MYSELF\myself:(M,WDAC,WO) DESKTOP-MYSELF\None:(RX,WD,AD,WA) Everyone:(RX)

ipilcher commented 3 years ago

@marcello-pietrobon Can you provide a bit more detail on the process you used to build the driver?

For example, I haven't been able to get VisualStudio installed, as I have no idea which "workloads" I need to install. And assuming that I'm able to get the driver built, I'm not at all clear on how to install the updated driver; you wrote that you copied the generated files into the VioFs directory, but I don't have any directory by that name in my Windows 10 guest. Also, don't I have to sign the drivers somehow in order to install them?

YanVugenfirer commented 3 years ago

Hi @ipilcher The build environment instalaltion instructions are here: https://github.com/virtio-win/kvm-guest-drivers-windows/wiki/Building-the-drivers-using-Visual-Studio-2017

How to test sign during the build: https://docs.microsoft.com/en-us/windows-hardware/drivers/develop/signing-a-driver-during-development-and-testing

How to enable test signing on Windows: https://docs.microsoft.com/en-us/windows-hardware/drivers/install/the-testsigning-boot-configuration-option (tl;dr: run "Bcdedit.exe -set TESTSIGNING ON" with administration privileges and reboot).

daiaji commented 3 years ago

@marcello-pietrobon Thank you, I will test it later.

ipilcher commented 3 years ago

I was finally able to work through the process of building and signing the driver. My initial smoke testing seems to confirm that the latest code fixes this problem.

Could we please get a signed release with this fix included? The process to build and sign the driver is quite involved (not to mention the need to put the guest into "test mode" in order to use a self-signed driver).

marcello-pietrobon commented 3 years ago

Sorry I couldn't reply sooner.

I'm waiting too for a new release with this problem fixed, as meanwhile I have to run the Win VM with "Bcdedit.exe -set TESTSIGNING ON"

I used very simple and partial instructions in order to save time and trouble, so if someone else has to go through this...

The instructions here are an addition to those I wrote few posts above here.

The source code is in c, so you just install the standard C/C++ part of the Visual Studio Community edition installer (the 'Desktop development with C++' tab in the picture you see in https://docs.microsoft.com/en-us/cpp/build/vscpp-step-0-installation?view=msvc-160 ) and you also make sure to install the latest 'Windows 10 SDK' that you see in there (or other Windows version you are using).

Make also sure to go on the 'Individual Components' tab of the installer and select 'Windows Driver Kit'.

Now you are set to load the viofs/viofs.sln solution and compile the viofs project in there.

The trick that saves you a lot of time is to just compile the viofs part of the kvm-guest-drivers-windows code (that is not the entire library) by using the viofs/viofs.sln solution that you find in there, as just said.

On top of Visual Studio you can clearly see and choose the Platform and Configuration you need to compile.

Once compiled it appears an 'install' directory under the viofs project, and with the files in there you overwrite those of the original installation (backup them first).

At this point, I didn't care signing the drivers, I just used the command "Bcdedit.exe -set TESTSIGNING ON"

It's even simpler than it looks like written here.

Hope this helps,

If somebody wants I can attach the binaries I generated for myself. I didn't change any line of code of course but I don't want to take any responsibility for any problem whatsoever, there shouldn't be any though.

M

TecEssence commented 3 years ago

@YanVugenfirer , hello What is the release cycle of the virtio-win installation package? can the new version of the installation package be released as soon as possible? The latest on (https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/) is still the version in November 2020

YanVugenfirer commented 3 years ago

@TecEssence we are building new packages based on the added features and fixed bugs.

I will check how soon a new version can be uploaded.

Best regards, Yan.

TecEssence commented 3 years ago

@YanVugenfirer ,hello

We look forward to the new virtio-win installation package, when will it be released?

Best regards, Qing.

YanVugenfirer commented 3 years ago

Hello,

We are expecting the package during next two week.

Best regards, Yan.

fubar-1 commented 3 years ago

Hello @YanVugenfirer,

Has there been any progress on releasing an updated & signed package with bug fixes?

Thanks, Dave

YanVugenfirer commented 3 years ago

Hello @Dave--G,

I am pushing the release of the package. There is a problem with a signature now. MS retired cross-signed certificates, and we are working to attestation sign the release. Most probably the next release will be unsigned, yet.

Best regards, Yan.

fubar-1 commented 3 years ago

Thanks for the update @YanVugenfirer,

This is unfortunate. I understand the certificate matter is out of your hands. This sounds like a significant development. What is virtio-win's long term strategy to resolve this issue? Can you point us to where this is being publicly discussed?

YanVugenfirer commented 3 years ago

@Dave--G

We are working in two directions:

  1. Automating attestation singing for Windows 10. This way Windows 10 drivers will be signed with attestation signing provided by Microsoft and could be installed as the drivers previously signed with a cross-signing certificate.
  2. Try to influence Microsoft to reverse their policy for older OSes.

Here are some of the public discussions:

oosoG commented 3 years ago

I'm also suffering from this bug. Because it seems that there are only certification problems for older versions of Windows. For the moment, can't someone create the virtiofs.exe driver (+viosoc?) for WINDOWS 10 ? (eventually publishing them on a different location)

oosoG commented 3 years ago

While waiting for better. Isn't there someone who can share the compiled files? (signed, testsigned or not signed) @marcello-pietrobon @ipilcher ?

YanVugenfirer commented 3 years ago

@oosoG unsigned drivers build from the current master for Windows 8 and Windows 10 32 and 64bit: https://www.dropbox.com/s/x86xam26uuuc8z7/VirtioFS-Install.zip?dl=0

oosoG commented 3 years ago

Thank you @YanVugenfirer for sharing Its working (question: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/590)

fubar-1 commented 3 years ago

Thanks @YanVugenfirer for the unsigned build. This will save some time in setup testing this evening.

Can you give us any expectation for when the next signed build will be available?

jwatt commented 3 years ago

It looks like the contents of https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/ got an update a few days ago, and after installing that in Windows the VirtIO FS Device driver has version 100.85.104.20400 and says it's signed by 'Microsoft Windows Hardware Compatibility Publisher'.

ilmari-lauhakangas commented 3 years ago

I updated the driver to the latest and while I can rename and modify files, git does not work. My use case is bisecting LibreOffice regression bugs. I can't launch executables, I can't do checkouts. It might initialise some files and then fall on its face.

If I do

git clone -o bundle win32-5.0.git-bundle win32-5.0

I get

fatal: remote did not send all necessary objects

Can others reproduce the problem with git? I am using git 2.29.2.windows.2 under Powershell 7. I also updated WinFSP to v1.9.

Simplest test:

PS Z:\> git init test
fatal: cannot copy 'C:/Program Files/Git/mingw64/share/git-core/templates/hooks/commit-msg.sample' to 'Z:/test/.git/hooks/commit-msg.sample': No such file or directory

Shared dir is defined as

<filesystem type="mount" accessmode="passthrough">
  <driver type="virtiofs"/>
  <source dir="/home/user/libobugs"/>
  <target dir="libobugs"/>
  <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
</filesystem>
arthurrasmusson commented 2 years ago

I am also experiencing this bug on Windows 10 LTSC on an Arch Linux host. The virtiofs.exe Windows service appears to crash during some operations and others refuse the ability to write or delete files - I am able to create files and sometimes running executables works however sometimes it does not - possibly in cases where the .exe wants to modify something. I've tried changing file permissions in the shared directory to 777 to see if that might mitigate the problems and the socket is in group KVM. I'm not clear on if what I am experiencing is the same very same issue but on the latest public drivers I installed yesterday this issue appears to still exist.

Striffly commented 2 years ago

I also have problems when I perform operations with the virtio-fs shared folder :

List of actions made inside guest :

Host config :

Guest config :

YanVugenfirer commented 2 years ago

@viktor-prutyanov Please take a look

viktor-prutyanov commented 2 years ago

Hi @Striffly,

Thank you for detailed description.

Copy file from shared folder to local folder : fail, crash service Duplicate file inside shared folder (copy in same place inside shared folder) : fail, crash service

Could you please try adding queue="1024" to your libvirt XML as follows:

...
<filesystem type="mount" accessmode="passthrough">
  <driver type="virtiofs" queue="1024"/>
  ...
</filesystem>
...

If it doesn't work, could you please share your libvirt virtiofs-related XML and clarify, what was the approximate size of the file you copying?

viktor-prutyanov commented 2 years ago

Hi @arthurrasmusson,

I am able to create files and sometimes running executables works however sometimes it does not - possibly in cases where the .exe wants to modify something.

Could you please clarify, what executables are you using? If these are some private things, have you ever had such a problem with some common executables or even with standard command-line utilities? For now, there is very little information. I can only suggest you to add queue="1024" (as described above) if it hasn't already been done.

ilmari-lauhakangas commented 2 years ago

@viktor-prutyanov see my comment from August. Are you able to run 'git init' in a shared folder without problems?

viktor-prutyanov commented 2 years ago

Hi @ilmari-lauhakangas,

Sometimes command

Z:\> git init test

reproduces the error you got. But sometimes it works.

viktor-prutyanov commented 2 years ago

@ilmari-lauhakangas, PR https://github.com/virtio-win/kvm-guest-drivers-windows/pull/719 is supposed to fix your problem with git init

viktor-prutyanov commented 2 years ago

Hi @Striffly, @arthurrasmusson,

If your problems are still actual, I suggest you to open separate issues with your problems. I have to write this because current issue seems to be solved and should be closed.

agrisvv commented 2 years ago

i can say some more comments: work:

dose not work:

no problems with linux guest. event with 770 perms on files. no changes with DistinctPermsForSameOwnerGroup 1 on windows file perms shows as everyone read only and s-1-5-0 as modify.

env:

viktor-prutyanov commented 2 years ago

Hi @agrisvv,

Could you please open new issue?

Also, please add information about your host filesystem.

ilmari-lauhakangas commented 2 years ago 
* virtio package 215

This does not yet include the fix. Would be nice to get an updated ISO. I haven't tested the fix either.

viktor-prutyanov commented 2 years ago 
* virtio package 215

This does not yet include the fix. Would be nice to get an updated ISO. I haven't tested the fix either.

You can build the driver by yourself. It became much easier with the advent of EWDK: https://github.com/virtio-win/kvm-guest-drivers-windows/wiki/Building-the-drivers-using-Windows-11--EWDK

arthurrasmusson commented 2 years ago

@viktor-prutyanov

Could you please clarify, what executables are you using?

I have GZDoom and Steam games in the shared path: https://www.zdoom.org/downloads

arthurrasmusson commented 2 years ago

@viktor-prutyanov

I can only suggest you to add queue="1024"

I am not currently using Libvirt - I'm using LibVF.IO. What does this correspond to in raw QEMU commands?

viktor-prutyanov commented 2 years ago

@arthurrasmusson, queue-size=1024. For example:

-device vhost-user-fs-pci,chardev=chr-vu-fs0,queue-size=1024,tag=mount_tag,bus=pci.6,addr=0x0
arthurrasmusson commented 2 years ago

@viktor-prutyanov

queue-size=1024. For example:

-device vhost-user-fs-pci,chardev=chr-vu-fs0,queue-size=1024,tag=mount_tag,bus=pci.6,addr=0x0

Since upgrading packages on Arch I cannot get the VM to start. I'm now using the following parameters on start: -mem-path /dev/hugepages -object memory-backend-memfd,id=mem,size=8G,share=on -numa node,memdev=mem -chardev socket,id=char0,path=/home/user/.local/libvf.io/sockets/virtio-fs-001.sock -device vhost-user-fs-pci,chardev=char0,tag=lime,queue-size=1024

On start virtiofsd crashes on the host with the following error message: "virtio_loop: Unexpectted poll revents 11"

The host is Arch Linux.

Is VirtIO-FS currently considered stable for use with Windows guests?

viktor-prutyanov commented 2 years ago

Since upgrading packages on Arch I cannot get the VM to start.

What is your QEMU version?

On start virtiofsd crashes on the host with the following error message: "virtio_loop: Unexpectted poll revents 11"

On virtiofsd start or on VM start?

Is VirtIO-FS currently considered stable for use with Windows guests?

No, it is a tech preview.

arthurrasmusson commented 2 years ago

@viktor-prutyanov

What is your QEMU version?

6.2.0

On virtiofsd start or on VM start?

On VM start.

xiagao commented 2 years ago

@viktor-prutyanov

queue-size=1024. For example:

-device vhost-user-fs-pci,chardev=chr-vu-fs0,queue-size=1024,tag=mount_tag,bus=pci.6,addr=0x0

Since upgrading packages on Arch I cannot get the VM to start. I'm now using the following parameters on start: -mem-path /dev/hugepages -object memory-backend-memfd,id=mem,size=8G,share=on -numa node,memdev=mem -chardev socket,id=char0,path=/home/user/.local/libvf.io/sockets/virtio-fs-001.sock -device vhost-user-fs-pci,chardev=char0,tag=lime,queue-size=1024

On start virtiofsd crashes on the host with the following error message: "virtio_loop: Unexpectted poll revents 11"

Seems the vm failed to start, the virtiofsd quit with "virtio_loop: Unexpected poll revents 11", so do you have any error msg from qemu-kvm?

The host is Arch Linux.

Is VirtIO-FS currently considered stable for use with Windows guests?

viktor-prutyanov commented 2 years ago

I have to close this issue because the original problem seems to be solved. Please open new separate issues with your problems, as I previously suggested.