How does this work?

[DUOLabs333]

I'm looking to do something similar (communicate over a shared block device), but I couldn't figure out a way to synchronize reads/writes without using an auxiliary communication channel (like an existing TCP connection).

[DUOLabs333]

If I switch to using regular files on the host, I no longer get the "not established" message, but both sides are still offline.

[fiddyschmitt]

Hmm try deleting and recreating the files. The stable and experimental builds use the files quite differently, and the existing content could be interfering.

[DUOLabs333]

I zeroed out the files before starting again --- same problem.

[fiddyschmitt]

So the experimental build does not work on the 9p shared files, nor the --drive files?

[DUOLabs333]

Yes, it does not work.

[fiddyschmitt]

Okay I'll set up QEMU on my machine tomorrow. Busy all today

[fiddyschmitt]

Hey,

On my setup (Debian 12 Host, Debian 12 Guest) both virtiofs and virtio-9p shared folders work fine, using stable & preallocated versions.

I created the VM using Virtual Machine Manager, and it launched this process:

/usr/bin/qemu-system-x86_64 -name guest=debian11,debug-threads=on -S -object {"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-10-debian11/master-key.aes"} -machine pc-q35-7.2,usb=off,vmport=off,dump-guest-core=off,memory-backend=pc.ram -accel kvm -cpu host,migratable=on -m 2048 -object {"qom-type":"memory-backend-memfd","id":"pc.ram","share":true,"x-use-canonical-path-for-ramblock-id":false,"size":2147483648} -overcommit mem-lock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 52c1ed32-7d37-49f8-9dbb-3fc518fe99c3 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=34,server=on,wait=off -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device {"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"} -device {"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"} -device {"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"} -device {"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"} -device {"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"} -device {"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"} -device {"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"} -device {"driver":"pcie-root-port","port":23,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x7"} -device {"driver":"pcie-root-port","port":24,"chassis":9,"id":"pci.9","bus":"pcie.0","multifunction":true,"addr":"0x3"} -device {"driver":"pcie-root-port","port":25,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x3.0x1"} -device {"driver":"pcie-root-port","port":26,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x3.0x2"} -device {"driver":"pcie-root-port","port":27,"chassis":12,"id":"pci.12","bus":"pcie.0","addr":"0x3.0x3"} -device {"driver":"pcie-root-port","port":28,"chassis":13,"id":"pci.13","bus":"pcie.0","addr":"0x3.0x4"} -device {"driver":"pcie-root-port","port":29,"chassis":14,"id":"pci.14","bus":"pcie.0","addr":"0x3.0x5"} -device {"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"} -device {"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"} -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/debian11.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":false,"discard":"unmap","driver":"qcow2","file":"libvirt-2-storage","backing":null} -device {"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-2-format","id":"virtio-disk0","bootindex":2} -blockdev {"driver":"file","filename":"/home/smith/iso/clonezilla-live-3.1.2-22-amd64.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"} -device {"driver":"ide-cd","bus":"ide.0","drive":"libvirt-1-format","id":"sata0-0-0","bootindex":1} -chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-10-debian11/fs0-fs.sock -device {"driver":"vhost-user-fs-pci","id":"fs0","chardev":"chr-vu-fs0","tag":"sharedfolder1","bus":"pci.7","addr":"0x0"} -fsdev local,security_model=mapped,id=fsdev-fs1,path=/home/smith/for_debian_11_vm -device {"driver":"virtio-9p-pci","id":"fs1","fsdev":"fsdev-fs1","mount_tag":"sharedfolder2","bus":"pci.8","addr":"0x0"} -netdev {"type":"tap","fd":"35","vhost":true,"vhostfd":"37","id":"hostnet0"} -device {"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:67:a9:1c","bus":"pci.1","addr":"0x0"} -chardev pty,id=charserial0 -device {"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0} -chardev socket,id=charchannel0,fd=33,server=on,wait=off -device {"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"} -chardev spicevmc,id=charchannel1,name=vdagent -device {"driver":"virtserialport","bus":"virtio-serial0.0","nr":2,"chardev":"charchannel1","id":"channel1","name":"com.redhat.spice.0"} -device {"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"} -audiodev {"id":"audio1","driver":"spice"} -spice port=5900,addr=127.0.0.1,disable-ticketing=on,image-compression=off,seamless-migration=on -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"} -device {"driver":"ich9-intel-hda","id":"sound0","bus":"pcie.0","addr":"0x1b"} -device {"driver":"hda-duplex","id":"sound0-codec0","bus":"sound0.0","cad":0,"audiodev":"audio1"} -chardev spicevmc,id=charredir0,name=usbredir -device {"driver":"usb-redir","chardev":"charredir0","id":"redir0","bus":"usb.0","port":"2"} -chardev spicevmc,id=charredir1,name=usbredir -device {"driver":"usb-redir","chardev":"charredir1","id":"redir1","bus":"usb.0","port":"3"} -device {"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"} -object {"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"} -device {"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.6","addr":"0x0"} -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on1

[DUOLabs333]

Interesting --- it's the same setup I have (except for the virtiofs --- it's not available on macOS).

[DUOLabs333]

I think the reason the --drive option isn't working well with FT is because since both sides are drives, C# probably can't get the right size (if you try to seek to the end of the block device in C, you get a size of 0, regardless of how big the actual drive is).

UPDATE: Yeah, that's the problem. There's no good way to fix this other than using OS-specific methods.

[fiddyschmitt]

Interesting! I'll take a look tonight.

[DUOLabs333]

I switched back to using 9p (it turns out that trying to use drives directly is a recipe for disaster), and it's interesting that the host does notice when the guest attaches/detaches, but the guest can not detect the host.

[fiddyschmitt]

Interesting. 9p had high latency for me (800 ms) whereas virtiofs was fine (5ms). I'll get bandwidth measurements tonight

On Thu, 22 Aug 2024, 06:50 DUO Labs, @.***> wrote:

I switched back to using 9p (it turns out that trying to use drives directly is a recipe for disaster), and it's interesting that the host does notice when the guest attaches/detaches, but the guest can not detect the host.

— Reply to this email directly, view it on GitHub https://github.com/fiddyschmitt/File-Tunnel/issues/18#issuecomment-2302989366, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADVA3TCZ5QXKW5T6T4DGQOLZST4QJAVCNFSM6AAAAABMGRRNA2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBSHE4DSMZWGY . You are receiving this because you modified the open/close state.Message ID: @.***>

[DUOLabs333]

Sorry I haven't been responding for a while --- I was writing an alternative POC of the same idea (using files to share data). Now that I have a prototype working (though it's slower than plain TCP), I think I know why FT hasn't been working well with macOS <-> Linux --- it was on Linux's side, or more precisely, it was due to my setup. Since I mounted the files as drives in Linux, if a change is made on the drives by QEMU, Linux is not notified that something changed. Therefore, we have to manually notify the OS to check the file/drive for updates with posix_fadvise.

[fiddyschmitt]

Hi @DUOLabs333, I just released v2.2.3 which adjusts how Linux guests read the shared file. If able, could you please give it a try with your setup which wasn't getting sync? Thanks

[DUOLabs333]

Cool! I'll look at this again.

[DUOLabs333]

Running FileTunnel on Linux gives me SendPump: Invalid argument --- I'm guessing it's because C# can not open block devices.

[YourSandwich]

I don't have that issue. What command are you exactly running?

Running FileTunnel on Linux gives me SendPump: Invalid argument --- I'm guessing it's because C# can not open block devices.

I have just packaged ft for ArchLinux https://aur.archlinux.org/packages/ft-bin

[DUOLabs333]

@YourSandwich I'm running ./ft-linux-arm64 --read /dev/disk/by-id/virtio-conn-write --write /dev/disk/by-id/virtio-conn-read.

[YourSandwich]

Ok, yeah, it cannot read block devices.