VirtFS / 9p support for Windows

ildar commented 7 years ago

https://github.com/virtio-win/kvm-guest-drivers-windows/issues/60 notes that "Gal is working on it". I didn't find his repo. So let's track progress here! @hammerg , @YanVugenfirer invited ;)

hammerg commented 7 years ago

@ildar unfortunate "working on it" is a bit exaggerated. I made a working prototype using dokan few years ago. However I failed to take it to the next step and implement it as a file system driver. I'll get back to it but it will take time.

ildar commented 7 years ago

ok. let it be open for tracking then. Thanks.

krysoft commented 6 years ago

Any progress on this project? I'm using ftp sharing to guest and have not so good performance against virtualbox. I'm curious how can the 9p increase the performance. Can someone here post some info about it? Thanks.

hammerg commented 6 years ago

No updates yet. It is still in the queue.

Warrentheo commented 6 years ago

Is it possible to get the code for your proof of concept? I would like to try and help if possible...

daiaji commented 6 years ago

I'm looking forward to using Virtfs on Windows guest OS！

art-ist commented 5 years ago

@Warrentheo maybe you want to look into [this piece of code, I foiund](https://github.com/mifritscher/qemu/tree/556be221a307381ab6406c24f3ad76cd1dac1878/hw/9pfs in a qemu fork) by @mifritscher. @hammerg what do you think about it?

mifritscher commented 5 years ago

Attention: I tried to make 9p running on a Windows host, NOT on a Windows guest!

The biggest obstacle is that the variants of open(root-directory,directory in this root-directory) is needed for security. The "normal" Win32 API can't do this, but the ntdll API can luckily archive this. Is a bit of pain because of conversions and the functions need to be called by getProcAddress, but it is doable. I'll search for my test code this evening. It can open a directory, but has e.g. stack corruption problems.

vrozenfe commented 5 years ago

In my understanding a proper 9p fs solution for Windows should come in two pieces - some sort of bus driver (virtio device) and a network file system driver (so-called "monolithic redirector" driver, which with some modifications might be used for network fs over vsock as well). I might be able to start working on some sort of POC a bit later, if Gal doesn't mind of course.

art-ist commented 5 years ago

Hey great response times, folks! Maybe Dokany could help with the file system driver part?

mifritscher commented 5 years ago

ReactOS (Windows XP clone) could be interested in that as well. They added support for NFS and VirtualBox shared folders, both using the redirector infrastructure. Because ReactOS is open source this could ease the development - and at least the redirection code part of the NFS driver could be reused.

kvishnivetsky commented 5 years ago

I'm interested in this implementation.

Claw256 commented 5 years ago

Any update yet?

YanVugenfirer commented 5 years ago

Unfortunately, the project is on hold.

Claw256 commented 5 years ago

Ok, thank you for your work. Do you know when the project could be started on again?

YanVugenfirer commented 5 years ago

No... I guess the current option is an external contribution. We can try and guide implementation.

Claw256 commented 5 years ago

Ok, hopefully someone can pick this up in the future, thanks anyways.

MarcusWichelmann commented 5 years ago

To my understanding, Windows 10 can act as a 9P client to access files of its Windows Subsystem for Linux (WSL). So there will probably already be a 9P implementation in the Windows Kernel. Might there be some way to re-use that to use virtfs inside of a Windows guest?

More information: https://devblogs.microsoft.com/commandline/whats-new-for-wsl-in-windows-10-version-1903/

Claw256 commented 5 years ago

Just an update to this, with WSL2 (Announced yesterday, May 6th), the Linux kernel compatibility layer has been replaced by the real Linux kernel! Read up more about it here: https://devblogs.microsoft.com/commandline/announcing-wsl-2/

alphaonex86 commented 4 years ago

now exists virtio-fs, very better than virtio-9g

MarcusWichelmann commented 4 years ago

@alphaonex86 virtio-fs looks very cool. But since it's based on FUSE it probably won't work on Windows either, right?

YanVugenfirer commented 4 years ago

There is a fuse port to Windows. https://github.com/billziss-gh/winfsp

On Sep 15, 2019, at 20:05, Marcus Wichelmann notifications@github.com wrote:

@alphaonex86 virtio-fs looks very cool. But since it's based on FUSE it probably won't work on Windows either, right?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

sej7278 commented 4 years ago

virtio-fs seems a bit bleeding edge - its not even made it into any linux distro's yet, so windows is surely years away

laris commented 4 years ago

good feature to track here.

AngusThermo-Pyle commented 4 years ago

Virtio-fs appears to have been added: https://github.com/virtio-win/kvm-guest-drivers-windows/commit/e864b9b0f063311d174c35d210b4d5cd1725ddb8

This is likely due to it's inclusion into the 5.4 kernel.

idarlund commented 4 years ago

Virtio-fs appears to have been added: e864b9b

This is likely due to it's inclusion into the 5.4 kernel.

Do you know if the future Virtio-fs driver work with -virtfs device?

DocMAX commented 4 years ago

hmm the driver still isn't in the latest ISO?

idarlund commented 4 years ago

I had an open question about this in a seperate issue. Closed it as duplicate to this one. When will viofs be included as a signed driver in the virtio-win iso?

hammerg commented 4 years ago

I had an open question about this in a seperate issue. Closed it as duplicate to this one. When will viofs be included as a signed driver in the virtio-win iso?

It should be included in the next release in about 3 months.

daiaji commented 4 years ago

Sounds like there are virtio-FS Windows drivers available now, right?🥰

idarlund commented 4 years ago

Sounds like there are virtio-FS Windows drivers available now, right?smiling_face_with_three_hearts

where did you find it? it's not included in the virtio-win iso (https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/) yet

idarlund commented 4 years ago

0.1.185-2 was released today; https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.185-2/ - unfortunately without virtio-fs. I guess this is just a patch release for the 0.1.185.

vrozenfe commented 4 years ago

virtio-fs is not a part of build 185. It was initially included in build 186. I will try to build a new RPM and make it public soon, hopefully by end of the week.

mkdy commented 4 years ago

As I see, virtio-win-0.1.187-1 now contains 9p drivers in /viofs/ directory. Unfortunately, there is no support for win7 at the moment. Are there any plans to include w7 drivers? Thanks!

DocMAX commented 4 years ago

also interested in win7 support. but happy that finally released for win10. now i can create fake local drives with a linux filesytem. great for spoofing apps! (with upcomming -m switch, not yet included with current release)

YanVugenfirer commented 4 years ago

@hammerg Isn't the Win7 support can be solved by just adding the build configuration?

idarlund commented 4 years ago

I'm not getting the driver to work properly. I have added the virtfs in qemu like this: -virtfs local,path=/mnt/extra/,mount_tag=D,security_model=mapped \ And it makes a new device show up in windows; "PCI Device". When trying to install (even force install) the driver on this device, it still lists as "PCI Device" and not loading.

I will gladly help in troubleshooting this, but I'm not sure what to do next. Any advice?

EDIT: Found some events in event viewer that might help:

Device PCI\VEN_1AF4&DEV_1009&SUBSYS_00091AF4&REV_00\3&2411e6fe&0&18 was not migrated due to partial or ambiguous match.

Last Device Instance Id: PCI\VEN_1AF4&DEV_1052&SUBSYS_11001AF4&REV_01\3&2411e6fe&0&18
Class Guid: {745a17a0-74d3-11d0-b6fe-00a0c90f57da}
Location Path: PCIROOT(0)#PCI(0300)
Migration Rank: 0xF000FFFFFB010123
Present: false
Status: 0xC0000719

bingzhangdai commented 4 years ago

understanding, Windows 10 can act as a 9P client to access files of its Windows Subsystem for Linux (WSL). So there will probably already be a 9P implementation in the Windows Kernel. Might there be some way to re-use that to use virtfs inside of a Windows

Just a little bit confused. If latest Win10 already supports 9P, does that mean we do not need to manually install the 9P driver? I just think Microsoft can do it better because they know Windows the best and can do more in kernel.

DocMAX commented 4 years ago

Does anybody have a idea how to make the virtiofs.exe run automatically in background after Windows boot. A behavior like the standard network drives? Right now i have to open an extra CMD window and keep it open.

bingzhangdai commented 4 years ago

Does anybody have a idea how to make the virtiofs.exe run automatically in background after Windows boot. A behavior like the standard network drives? Right now i have to open an extra CMD window and keep it open.

@DocMAX Maybe you can use TaskScheduler? Remember to check "Run whether user is logged on or not".

FailSpy commented 4 years ago

virtio-latest from Fedora's downloads now includes these under the viofs file. (0.1.187+)

Using device manager, install them to the currently driverless storage device, and hopefully it will just install and thusly be identified as "Virtio FS Device"

Once installed, you'll need to run the virtiofs.exe with the device to be able to see the device as a network drive. I would recommend moving the whole folder for your architecture(e.g. viofs/w10/amd64) to some comfortably static location in your W10 VM. I put mine in C:\Program Files\viofs\

You'll need to install WinFSP for this if you don't already have it. Once installed, it may complain about still not being able to see 'winfsp-x64.dll'. As a work around, you can go find it in your install directory (C:\Program Files (x86)\WinFSP\bin) to get the DLL, and copy the needed DLL into the same folder as your virtiofs.exe

Once this is all done, you'll be able to launch virtiofs.exe and have a network drive. To get virtiofs.exe to run as a service with the system, you can use this command in an Administrator Command Prompt to create a custom service: sc create viofs binpath="(your binary location)\virtiofs.exe" type=own start=auto DisplayName="VirtioFS Driver"

This will add a Windows service that you can start and stop to mount the drive under the name of 'viofs', and you can set it to automatically start with your system. (Command included will start 'auto', change to 'demand' for manual start/stop from services list if you'd rather)

Now either restart, or start the service from your Services list manually after adding it. You should now be setup with a network drive with the mount_tag

DocMAX commented 4 years ago

I have stress tested this driver and i must say it's very unstable. I constantly get "qemu-system-x86_64: Failed to read from slave." on the command line. When this happens, only a reboot can bring back the drive in Windows. Please have a look! Edit: I saw that virtiofsd crashes with "Bad system call"

bingzhangdai commented 4 years ago

I just click the virtio-win-gt-x64.msi to install the driver. Am I wrong?

I am afraid I do not install the driver properly. When I start the virtiofs.exe, I get "The service VirtIO-FS has failed to start (Status=c0000008).". And in the device manager, it still shows as PCI device not "Virtio FS Device".

Marco98 commented 4 years ago

@bingzhangdai what is the hardware-id of your device? It should look like this:

If you are using libvirt the xml spec should look similar to this (driver type="virtiofs" is important):

<filesystem type="mount" accessmode="passthrough">
  <driver type="virtiofs"/>
  <source dir="/virtiofs"/>
  <target dir="virtiofs"/>
  <address type="pci" domain="0x0000" bus="0x0f" slot="0x00" function="0x0"/>
</filesystem>

FailSpy commented 4 years ago

If you're struggling with the driver installer, you can try to point Windows at it manually.

Find the device in device manager, and right click it. Choose 'Update driver', and in the window that appears, choose 'Browse my computer for driver software'

Then point it to the folder where you extracted the driver on your VM for it: e.g. 'viofs/win10/amd64', and hit next. It should auto detect the driver match, and install it to the device

@bingzhangdai

bingzhangdai commented 4 years ago

@Marco98 @FailSpy Thanks.

I use PVE and I manually added the following line in the vm config: args: -fsdev local,security_model=mapped,id=fsdev0,path=/root/mnt/sata0 -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=sata0 -fsdev local,security_model=mapped,id=fsdev1,path=/root/mnt/sata1 -device virtio-9p-pci,id=fs1,fsdev=fsdev1,mount_tag=sata1 ( I mount two sata drives). The Linux guest works fine but in Win 10, device manager shows,

Sadly, it does not show up as "VirtIO FS Device", If I update the driver by choosing "Browse my computer for driver software", It shows "Windows was unable to install you PCI Device" :(

Marco98 commented 4 years ago

@bingzhangdai i think you are not using the right device. You are using the device virtio-9p-pci. But the device for virtiofs is called vhost-user-fs-pci. The qemu parameters at my host are:

-chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-1-win2/fs0-fs.sock -device vhost-user-fs-pci,chardev=chr-vu-fs0,tag=virtiofs,bus=pci.15,addr=0x0

FailSpy commented 4 years ago

@bingzhangdai Seems that device 'virtio-9p-pci' isn't quite what the driver is expecting to work with? Don't quote me on that though. As a thing to try; Attempt it with vhost-user-fs-pci as your device, as defined here: https://qemu.readthedocs.io/en/latest/tools/virtiofsd.html

More detailed reference here: https://virtio-fs.gitlab.io/howto-qemu.html

I'm not getting the driver to work properly. I have added the virtfs in qemu like this: -virtfs local,path=/mnt/extra/,mount_tag=D,security_model=mapped \ And it makes a new device show up in windows; "PCI Device". When trying to install (even force install) the driver on this device, it still lists as "PCI Device" and not loading.

@idarlund Seems like you may have a duplicate issue to our friend just above here. -virtfs flag appears to be a sort of alias to the 9p-pci device rather than the device the driver is expecting vhost-user-fs-pci. If you're still having that issue, may want to try reviewing the above.

bingzhangdai commented 4 years ago

Thanks very much! It seems that I am using the wrong device.

I previously followed the doc Documentation/9psetup FailSpy pointed out.

It seems that, 9p is VirtFS and here, the driver is expecting me to use virtio-fs? VirtFS and virtio-fs are different. Is that correct? Linux supports both the two devices but on Windows, the driver could only use virtio-fs? I get confused with the name. orz

FailSpy commented 4 years ago

It seems that, 9p is VirtFS and here, the driver is expecting me to use virtio-fs? VirtFS and virtio-fs are different. Is that correct? Linux supports both the two devices but on Windows, the driver could only use virtio-fs? I get confused with the name. orz

I hope not, otherwise we're on the completely wrong thread O_O But I believe Virtio-FS and VirtFS are different names for the same architecture, just there's a divided way of implementing it based on the VM, though the specifics of how/why eludes me. I may of course be completely wrong about this at which point most of what's above is technically off-topic.

Edit: assuming that virtio-fs and virtfs are the same architecture as is said, virtfs is an offshoot from plan 9's protocol, which itself allows for networked filesystems, and virtfs is the project to adapt that into something different tuned for virtualization environments.

And that -virtfs flag in QEMU could be a legacy in the earlier days when virtfs just was closer to plan9 sharing? Though just guesswork at this point.

Post-Edit, my personal verdict: VirtFS is NOT Virtio-FS. But it does seem like for what people wanted 9-plan support for on a VM, this is a seemingly better drop-in replacement for them. But arguably, 9-plan mount support doesn't exist.

But Windows does have its own solutions for that which can be implemented now thanks to improvements in WSL. Whether KVM needs to support as a driver device seems like a different issue, and a less necessary one as Virtio-FS consumes most of its desire.

So I'd argue this is the correct place still to talk about Virtio-FS support as far as virtio-win project is concerned.

virtio-win / kvm-guest-drivers-windows

VirtFS / 9p support for Windows #126