Synchronization on Linux Platform

[zhexwang]

Hi, I saw that you need KeSignalCallDpcSynchronize() to launch vmx for each cpu core. Is it right? Do you know how to operate this on Linux Platform? Thank you very very much!

[asamy]

Actually this synchronization is DPC specific, in Linux you won't need this. IIRC it's just used in windows to tell the IPI caller it's done.

I think you can use for_each_possible_cpu under Linux, you might also need to disable preemption: preempt_disable().

[zhexwang]

@asamy Thank you very much for your response! I had tried the for_each_possible_cpu(), but it didn't work. Maybe smp_call_function() could solve this problem? And thanks for your suggestion (disable preemption).

[asamy]

What do you mean it didn't work? Can you show me how you're doing it? Yep, smp_call_function does an IPI, so it'll work and it sounds more reliable too.

On possible cpus loop, I think you can try this: [see below]

[zhexwang]

I had wrote the code below: preempt_disable(); for_each_possible_cpu(cpu){ // cpu_number times per_cpu_function(); } preempt_enable(); I found that the per_cpu_function() runs on the current cpu core with cpu_number times.

[asamy]

Yep, nvm about the get_cpu and put_cpu actually, these disable and enable preemption and give you the current SMP processor number. I think you'd want something like this:

static void vcpu_init(struct vcpu *vcpu) {
   /* do whatever  */
}

for_each_online_cpu(cpu)
       smp_call_function_single(cpu, vcpu_init, &ksm.vcpu_list[cpu], 1);  /* 1 is wait  */

[zhexwang]

WOW, thank you so much!

for_each_online_cpu() had already disabled and enabled preemption?

[asamy]

No, but I think you don't need to do that inside vcpu_init anyway, as you're pretty much called from the call-function IPI IRQ handler. What I'd recommend (if you're porting ksm that is), initialize ept per-cpu before calling vcpu_init as it can take time (on debug builds specially) so you don't block all CPUs for a long time, something like this:

/* Initialize EPT for each online CPU:  */
for_each_online_cpu(cpu) {
   struct vcpu *vcpu = &ksm.vcpu_list[cpu];
   if (!ept_init(&vcpu->ept))
         goto err;
}

/* Now virtualize each online CPU:  */
for_each_online_cpu(cpu)
   smp_call_function_single(cpu, vcpu_init, &ksm.vcpu_list[cpu], 1);

err:
   return -ENOMEM;

[zhexwang]

Got it! You are right! I had saw the source code of smp_call_function_single() #L248 http://lxr.free-electrons.com/source/kernel/smp.c