virtio-fs: More testing

kevinoid commented 3 years ago

In https://github.com/virtio-win/kvm-guest-drivers-windows/issues/527#issuecomment-754631925 @YanVugenfirer mentioned that some virtio-fs specific CI tests might be welcomed. I'd suggest considering the test suites listed in WinFsp Testing (excluding fscrash, which is not applicable):

[ ] winfsp-tests (in external mode) - Tests developed for Winfsp.
[ ] winfstest - Inspired by the FreeBSD fstest, but written from scratch to work on Windows.
[ ] Microsoft Installable File System Filter Test - Part of Windows Hardware Lab Kit (HLK). (Is this already run by AutoHCK?)
[ ] fsx - A port of Apple's fsx program.
[ ] fstorture - A port of Apple's fstorture program.
[ ] fsbench - A benchmarking tool which could be used for performance regression testing.

What would you think about applying some or all of these tools? For tools without binaries (winfstest, fsx, fstorture, fsbench) would it be preferable to contribute automated builds upstream or to build on your own CI?

Would it make sense to run each test on several host filesystems, either as separate viofs mounts or subdirectories of a single shared viofs mount? If so, which filesystems:

[ ] tmpfs
[ ] ext4
[ ] btrfs
[ ] zfs
[ ] xfs
[ ] ntfs-3g
[ ] vfat
[ ] f2fs

Is there a likely to be benefit from running a subset on read-only filesystems (isofs, udf, cramfs, squashfs)? My guess is that testing them would have poor cost/benefit.

I would expect network filesystems (nfs, cifs) to reveal oddities during testing. However, again, my guess is that testing them would have poor cost/benefit.

Has the behavior of overlayfs been considered? (e.g. non-unique st_ino, differing st_dev between directories/files, changing st_dev/st_ino after write due to copy-up) I suspect it would not benefit much from the tests above, but may benefit more from white-box testing depending on how st_dev/st_ino is used by viofs. May be worth considering later.

Thoughts? I can work up scripts to prepare the host mounts, download/extract test binaries on the guest, and run the test binaries on the guest, but that only goes so far. I would understand if you don't have the resources to justify integrating such tests with AutoHCK at the moment. (I also can't guarantee I will have time to get the scripts done quickly either.) Let me know if you'd prefer to start smaller, to leave this issue unimplemented for possible future work, or how you'd like to proceed.

Thanks for considering, Kevin

YanVugenfirer commented 3 years ago

Adding @meirav12 and @Basils to the discussion.

YanVugenfirer commented 3 years ago

Hi Kevin,

Thanks a lot for the elaborate description. I will add more people to the thread that can help us with testing discussion.

Best regards, Yan.

YanVugenfirer commented 3 years ago

Hi,

One comment on the Installable file system filter test - it should run on the filter driver. @hammerg is WinFSP a filter driver? On other hand, as far as I know, the creator of WinFSP runs HCK tests on his binaries.

HCK-CI is able already to run tests for filesystem minifilters.

Best regards, Yan.

lijinlijin commented 3 years ago

Adding virtio-fs QE @menli820 to this discussion.

xiagao commented 3 years ago

Hi Kevin, Do you know how to build winfsp-tests tool, I didn't get the method from the github repo.

BR, Xiaoling

kevinoid commented 3 years ago

Hi @xiagao,

I believe winfsp-tests is built from build/VStudio/testing/winfsp-tests.vcxproj (as part of build/VStudio/winfsp.sln) although I have not built it myself yet.

Let me know if there's anything I can do to help.

Thanks for looking into it! Kevin

xiagao commented 3 years ago

Hi @xiagao,

I believe winfsp-tests is built from build/VStudio/testing/winfsp-tests.vcxproj (as part of build/VStudio/winfsp.sln) although I have not built it myself yet.

Let me know if there's anything I can do to help.

Thanks for looking into it! Kevin Can build successfully, thank you. But hit some error when running it. Z:\build\build\Release>winfsp-tests-x64.exe --external --resilient create_test............................ OK 0.09s create_fileattr_test................... KO ASSERT(INVALID_HANDLE_VALUE != Handle) failed at create-test.c:275:create_fileattr_dotest

Z:\build\build\Release>winfsp-tests-x64.exe --external create_test............................ KO ASSERT(ERROR_FILE_EXISTS == GetLastError()) failed at create-test.c:49:create_dotest

Z:\build\build\Release>winfsp-tests-x64.exe fuse_opt_parse_test.................... OK 0.00s posix_map_sid_test..................... OK 0.00s posix_map_sd_test...................... OK 0.00s posix_merge_sd_test.................... OK 0.00s posix_map_path_test.................... OK 0.00s uuid5_test............................. OK 0.00s path_prefix_test....................... OK 0.00s path_suffix_test....................... OK 0.00s dirbuf_empty_test...................... OK 0.00s dirbuf_dots_test....................... OK 0.00s dirbuf_fill_test....................... OK 0.64s version_test........................... OK 0.00s launch_reg_test........................ OK 0.00s memfs_test............................. -->long time,6h, not sure it is correct

xiagao commented 2 years ago

Hi @xiagao, I believe winfsp-tests is built from build/VStudio/testing/winfsp-tests.vcxproj (as part of build/VStudio/winfsp.sln) although I have not built it myself yet. Let me know if there's anything I can do to help. Thanks for looking into it! Kevin Can build successfully, thank you. But hit some error when running it. Z:\build\build\Release>winfsp-tests-x64.exe --external --resilient create_test............................ OK 0.09s create_fileattr_test................... KO ASSERT(INVALID_HANDLE_VALUE != Handle) failed at create-test.c:275:create_fileattr_dotest

Z:\build\build\Release>winfsp-tests-x64.exe --external create_test............................ KO ASSERT(ERROR_FILE_EXISTS == GetLastError()) failed at create-test.c:49:create_dotest

@hammerg I still hit this issue, can you help to check? If you need the tool, I could provide. Thanks.

YanVugenfirer commented 2 years ago

@xiagao Thank you for reporting

@viktor-prutyanov Please take a look

YanVugenfirer commented 2 years ago

Hello All,

Please help us understanding you use cases for using virtio-fs, and thus make us virtio-fs support better. Please participate in the discussion and add your use cases: https://github.com/virtio-win/kvm-guest-drivers-windows/discussions/726

Thanks a lot, Yan.

virtio-win / kvm-guest-drivers-windows

virtio-fs: More testing #531