Page Fault when accessing PCI address

[adoehrman]

Hello, I’m running into an issue where I’m unable to read or write to a PCI device's memory from the hypervisor code. I use the base address of the PCI device, confirmed from UEFI, and pass an offset and variable to the hypervisor using a VMCALL, then attempt to dereference it at the passed offset as seen in the pseudocode snippet below:

static unsigned long* PciPhysPtr;
#define BAR1 0xE0100000
PciPhysPtr = (unsigned long *)BAR1;
uintptr_t control = this->rbx();
switch( control ) {
    case OPCODE_HYP_PCI_READ : // pci read
    {
      bfdebug_info(0, "Reading PCI...\n");

      MemReadData = *(PciPhysPtr + this->rcx());   // offset

      this->set_rax((uint64_t) MemReadData);
      break;
    }

    case OPCODE_HYP_PCI_WRITE : // pci write
    {
      bfdebug_info(0, "Writing PCI...\n");
      MemWriteData =  this->rdx();        

      *(PciPhysPtr + this->rcx()) = MemWriteData;

      // return some verification that it wrote
      this->set_rax((uint64_t) *(PciPhysPtr + this->rcx()));
      break;
    }
}

I test this with a VMCALL from a UEFI driver after launching the hypervisor, as seen in the next code snippet:

  a = 0;
  b = 11; // opcode
  c = 0xD000; // offset
  __asm__ __volatile__ (
      "push %%rax \n"
      "mov %0, %%rax \n"
      "mov %1, %%rbx \n"
      "mov %2, %%rcx \n"
      "mov %3, %%rdx \n"
      "vmcall \n"
      "pop %%rbx \n"
      "mov %%rax, %0 \n"
      : "=rm" (a)
      : "rm" (a), "rm" (b), "rm" (c), "rm" (d)
      : "memory", "cc", "rax", "rbx", "rcx", "rdx"
  );

When I attempt to read or write to this PCI space, I get this output from the hypervisor:

Am I missing some additional setup required before attempting to access memory over PCI, or memory in general from the hypervisor? Thanks, Austin

[rianquinn]

You cannot do that. The address that you have is not a virtual address owned by the hypervisor. Like anything else in the system, the hypervisor uses paging, which means anytime you dereference a pointer, that pointer must have an associated entry in the hypervisor's page tables telling it which physical address it should be reading.

To do that, you will need first make sure that you have a Guest Physical Address (GPA) and then you can map that GPA to an address that the hypervisor can dereference using: https://github.com/Bareflank/hypervisor/blob/d2203d8fa44339e1c7dd0ce8264f5248b43a7648/bfvmm/include/hve/arch/intel_x64/vcpu.h#L1579

[adoehrman]

The mapping function seems to have been what I was missing. Thanks for the help!