wbenny / hvpp

hvpp is a lightweight Intel x64/VT-x hypervisor written in C++ focused primarily on virtualization of already running operating system
MIT License
1.11k stars 220 forks source link

Implement simple example of syscall interception #8

Open wbenny opened 6 years ago

rianquinn commented 6 years ago

I would be interested in how you approach this as there is no trapping support for Intel WRT the syscall instruction. There are ways to do it, but they are not pretty.

wbenny commented 6 years ago

I'm thinking about approach which sets custom MSR_LSTAR and returns the original MSR_LSTAR on RDMSR. That will require writing custom syscall handler. I didn't peek yet into how much work will it take or how ugly solution will that be.

ghost commented 6 years ago

I don't think you can do MSR_LSTAR hooks unless you somehow disable meltdown patch(or expose your handler to all UM processes, and on every new process creation). The reason for this is because they separate the kernel/usermode address space, and map entire KVASCODE section in ntoskrnl into all usermode processes I think. https://www.fortinet.com/blog/threat-research/a-deep-dive-analysis-of-microsoft-s-kernel-virtual-address-shadow-feature.html

I've tried setting MSR_LSTAR many times, but it results in instant BSOD in some random process due to page fault(which i'm assuming is because of the meltdown patch). The code I tested was just a simply jump to the original address that had been stored in MSR_LSTAR.

Probably best to just stick with EPT shadowing.

DebugBuggin commented 5 years ago

you can use this project for reference for hooking syscalls, I couldn't get hyperbone to load for me but the author is a genius and his code is very clean and minimum https://github.com/DarthTon/HyperBone

wbenny commented 5 years ago

Resurrecting this thread after I read this post: https://revers.engineering/syscall-hooking-via-extended-feature-enable-register-efer/

Basically, by disabling EFER.SCE flag, you'll get #UD on syscall/sysret instructions, which you can trap and emulate in the hypervisor.

It is not a new technique, it has been already used and described e.g. by Nitro, SecVisor, and I'm pretty sure I've seen it in few other papers too in the past. That post just made me realize I have this issue here hanging.

Although I'm not in urgent need to have this feature implemented, I leave it here when that time comes.

hzqst commented 5 years ago

redirecting RIP to custom handler at cr3 switching moment is fine when KvaShadow is enabled. something like:

            // The MOV to CR3 does not modify the bit63 of CR3. Emulate this
            // behavior.
            // See: MOV to/from Control Registers
            UtilVmWrite(VmcsField::kGuestCr3, (*register_used & ~(1ULL << 63)));

        ULONG_PTR JmpTo = (ULONG_PTR)IDT::GetCr3SwitchTrampoline((PVOID)guest_context->ip);
                if (JmpTo)
                {
                    const auto exit_inst_length = UtilVmRead(VmcsField::kVmExitInstructionLen);
                    UtilVmWrite(VmcsField::kGuestRip, JmpTo + exit_inst_length);
                    return;
                }
    PVOID GetCr3SwitchTrampoline(PVOID LinearAddress)
    {
        if (m_Cr3SwitchHooked)
        {
            if (m_HookContext[m_Cr3SwitchHookIndex].KvaCr3Switch == LinearAddress && m_HookContext[m_Cr3SwitchHookIndex].KvaCr3SwitchTrampoline)
                return m_HookContext[m_Cr3SwitchHookIndex].KvaCr3SwitchTrampoline;
        }
        return NULL;
    }
            m_Cr3SwitchHooked = true;
            m_Cr3SwitchHookIndex = index;

            SIZE_T trampoSize = (PUCHAR)m_HookContext[index].KvaJmpToRealEntry - (PUCHAR)m_HookContext[index].KvaCr3Switch;

            PUCHAR trampoCode = (PUCHAR)ExAllocatePool(NonPagedPool, trampoSize + 6 + 8);
            RtlFillBytes(trampoCode, trampoSize + 6 + 8, 0xCC);
            RtlCopyMemory(trampoCode, m_HookContext[index].KvaCr3Switch, trampoSize);
            RtlCopyMemory(trampoCode + trampoSize, "\xFF\x25\x00\x00\x00\x00", 6);
            *(ULONG_PTR *)((PUCHAR)trampoCode + trampoSize + 6) = (ULONG_PTR)NewCode;
            m_HookContext[index].KvaCr3SwitchTrampoline = trampoCode;

111

ajkhoury commented 5 years ago

redirecting RIP to custom handler at cr3 switching moment is fine when KvaShadow is enabled. something like:

          // The MOV to CR3 does not modify the bit63 of CR3. Emulate this
          // behavior.
            // See: MOV to/from Control Registers
          UtilVmWrite(VmcsField::kGuestCr3, (*register_used & ~(1ULL << 63)));

      ULONG_PTR JmpTo = (ULONG_PTR)IDT::GetCr3SwitchTrampoline((PVOID)guest_context->ip);
              if (JmpTo)
              {
                  const auto exit_inst_length = UtilVmRead(VmcsField::kVmExitInstructionLen);
                  UtilVmWrite(VmcsField::kGuestRip, JmpTo + exit_inst_length);
                  return;
              }
  PVOID GetCr3SwitchTrampoline(PVOID LinearAddress)
  {
      if (m_Cr3SwitchHooked)
      {
          if (m_HookContext[m_Cr3SwitchHookIndex].KvaCr3Switch == LinearAddress && m_HookContext[m_Cr3SwitchHookIndex].KvaCr3SwitchTrampoline)
              return m_HookContext[m_Cr3SwitchHookIndex].KvaCr3SwitchTrampoline;
      }
      return NULL;
  }
          m_Cr3SwitchHooked = true;
          m_Cr3SwitchHookIndex = index;

          SIZE_T trampoSize = (PUCHAR)m_HookContext[index].KvaJmpToRealEntry - (PUCHAR)m_HookContext[index].KvaCr3Switch;

          PUCHAR trampoCode = (PUCHAR)ExAllocatePool(NonPagedPool, trampoSize + 6 + 8);
          RtlFillBytes(trampoCode, trampoSize + 6 + 8, 0xCC);
          RtlCopyMemory(trampoCode, m_HookContext[index].KvaCr3Switch, trampoSize);
          RtlCopyMemory(trampoCode + trampoSize, "\xFF\x25\x00\x00\x00\x00", 6);
          *(ULONG_PTR *)((PUCHAR)trampoCode + trampoSize + 6) = (ULONG_PTR)NewCode;
          m_HookContext[index].KvaCr3SwitchTrampoline = trampoCode;

111

But that means you gotta set the CR3 load exiting bit in the proc based controls vmcs field. You're gonna suffer some pretty big performance hits since you'll have to exit on every MOV to CR3 instruction. :(

Best solution is to find some way to do this without exiting. I have a solution by manually adding pages to the shadow page tables, but the implementation is pretty heavy and relies on a bunch of undocumented stuff, which is why I favor using the EFER MSR hook.