rust-vmm / vfio

14 stars 20 forks source link

Fix endianess for fd's #53

Closed juru1234 closed 1 month ago

juru1234 commented 1 month ago

Passing an fd from user-space to the kernel requires native endianess. Change LittleEndian to NativeEndian to also support big endian machines.

@epilys I think commit 07fd7ef3a1c2 ("vfio/pci: Use endian neutral helpers") is only tangentially related and as the commit message states did not change the observed behavior. As I understand it, that commit is about MMIO via pread() from the device fd which keeps the endianess of the read unchanged. This makes sense since it keeps the observed data consistent between pread() from the device fd and mmap() of the same fd where a load is then MMIO which for PCI is always little endian. For the case I encountered, this is about passing an fd from user-space to the kernel which requires native endianess.

epilys commented 1 month ago

@juru1234 yes, it's the commit message that stood out to me:

The iowriteXX/ioreadXX functions assume little endian hardware and convert to little endian on a write and from little endian on a read

But yeah it was just an uneducated guess!

epilys commented 1 month ago

I can't edit the PR description, so if someone could please close #53 when this is merged

liuw commented 1 month ago

@juru1234 can you please add vfio-ioctls: as the prefix in the subject line?

juru1234 commented 1 month ago

@liuw done

juru1234 commented 1 month ago

@liuw Thank you! Do you plan to make a new release for vfio-ioctls?