Disable Interrupt in vcpu_handle_exit()

[zhexwang]

I am trying to transplant ksm to linux platform. I had a question that why you didn't disable the interrupt in vcpu_handle_exit()? When I tried to disable the interrupt in my codes (linux-version of ksm), the os crashed and had no response!

[asamy]

Because at the time of VM exit (aka entry to __vmx_entrypoint in assembly), the CPU had already disabled all interrupts, so disabling them again or even raising TPR is pointless.

It works fine without explicit disable though?

[zhexwang]

Yes. It works fine without explicit disable.

[zhexwang]

I am sorry that I had another question that I didn't understand ksm->kernel_cr3 and ksm->origin_cr3. If I need to allocate the new page table for VMM (host-state) or only use the current CR3 for VMM when setting the vmcs? I found that when I upload the kernel module, the current CR3 belongs to insmod process. And the insmod process will exit soon, its page table will be free.

[asamy]

At the time of driver load, we use whatever the current cr3 (->origin_cr3) happens to be, and use it as the guest one because that's where we're going to return to so we don't crash the caller. ->kernel_cr3 is for the host, you use whatever the kernel uses.

It doesn't really matter whether insmod closes or not, the guest cr3 will be updated anyway with whatever gets scheduled, but if you use it's cr3 for the host, then it could be a potential problem.

[zhexwang]

I don't understand the windows platform. The ksm->origin_cr3 is only set before calling STATIC_CALL_DPC(__call_init, &ksm)! I think ksm->origin_cr3 only recording CR3 of the current cpu core. But other cpu cores' CR3 may be not this value. I think each cpu core should have its own origin CR3, is it right?

[asamy]

Because the DPC is actually an interrupt and therefore is called inside the kernel, so saving the "origin cr3" before it is essential, as reading cr3 inside the DPC will read the kernel's.

No, cr3 of a process (whether kernel or user) is not per cpu, it's shared. I think you just were confused that we read the CR3 everytime in the DPC (aka per-cpu), but nah, all of them read the same.

[zhexwang]

I know that a process has only one cr3. I mean that each cpu core may run different processe and had different cr3 value. At the same time, only one cpu core load the driver and ksm->origin_cr3 belongs to this core. But other cores may not had the same value of CR3. I thought the guest must return to its origin process after vmlaunch for each core. So if we should set the guest-state (espeically the CR3 register) based on current execution state for each core?

[zhexwang]

When I active vmx on all cpu cores by using smp_call_function_single(vcpu_init), I found that the CR3 values are different on these cores. I read the current CR3 in vcpu_init() and use it to set guest CR3. I allocate a new host page table and set the page pointer to Host CR3. Am I right?

[asamy]

For the origin cr3 case, yeah, it'd be a good idea to do that.

I am not sure I am understanding what you mean by allocating new host page table and set the page pointer to host cr3.

[zhexwang]

For the origin cr3 case, I also think there is no problem that the guest cr3 is set to ksm->kernel_cr3 (read in DPC). The guest will return to the origin execution state (all cores) naturally that is ensured by DPC.

[zhexwang]

For the host page table, I means allocate a new page table for VMM. This new page table exists enduringly! If the VMM's CR3 points to a existing kernel or user process's page table, there may exist a potential problem that the dependent process may be killed or finished and its page table is free.

[asamy]

There is no real reason to do that, just re-using the kernel cr3 is OK, for more safety it'd be fine to do former but I doubt it'd be needed.

[zhexwang]

Are you telling the host page table case or origin cr3 case?

[asamy]

Host.

[zhexwang]

I agree with you. What about the origin CR3 case ? Am I right?

[asamy]

Yes, that's correct. I can't tell you for sure right now what can be done on Linux, as I am really tired, you might want to do it manually by reading cr3 (unreliably) or maybe something in SMP will do it for you.

[zhexwang]

I am sorry for bothering you. I really appreciate your helps. Thanks again!

[asamy]

BTW, if you want I can give you push access to the repository and you can push to your own branch while porting, that way I can give you feedback, help you test, etc, and when it's stable it can get merged to master.

[zhexwang]

Hi, I had already solved all problems. And my hypervisor runs very well! Thank you very much!

[asamy]

You're welcome :) Patches always welcome!