optimize MPU context switch on ARM/ARC

[andrewboie]

There is some overhead on context switch due to reprogramming memory protection hardware. We always will need to update privilege mode stack guards, but user stack regions and memory domains can be skipped in some scenarios. Implement lazy MPU/MMU reprogramming such that:

• Reprogramming of thread stack MPU regions is avoided if the incoming thread is a supervisor thread
• Switching memory domains is skipped if the incoming thread is a supervisor thread or in the same memory domain.

This should span multiple context switches. Consider user threads A and B in the same memory domain X, and thread C in memory domain Y. Let S represent some context switch activity between any number of arbitrary supervisor threads.

• Context switching between A -> C updates memory domain and user stack info. We can't do anything lazily here.
• Context switching between A -> B should not require any memory domain reprogramming. Stack regions are updated. Some arches already do this.
• Context switching between A -> S -> B: should not require any memory domain reprogramming. User stack regions are only updated on the S -> B transition.
• Context switching between A -> S -> C: The switch to A -> S should not require any reprogramming. The S -> C transition should move from A's domain X to C's domain Y and update user stack regions.
• Context switching between A -> S -> A: Nothing to do.
• Context switching between S -> S: Nothing to do.

Optimistically I hope we can do this in some common code that calls into arch apis, rather than having to implement separately for each.

For implementation, we could perhaps add two (per-cpu?) fields to z_kernel:

struct k_mem_domain *last_domain;
k_thread_stack_t last_stack;

Upon context switching out, if the outgoing thread is in user mode, save the current memory domain and the thread's stack object pointer.

Upon context switching in, if the incoming thread is in user mode, then compare the incoming thread's memory domain and stack object with the value saved in the kernel, if they aren't the same take the respective actions to switch domains or update user stack region.

Any PRs will need profiling data to show that this actually improves performance.

[ioannisg]

I am not sure how critical this is for ARM, since context-switch executes on lowest priority, anyway. :) There's value here, of course.

[andrewboie]

I am not sure how critical this is for ARM, since context-switch executes on lowest priority, anyway. :) There's value here, of course.

@ioannisg in my opinion, context switch needs to be the most tightly optimized code in the system..

This is now implemented on x86, so scope of this ticket is for ARM/ARC

[andrewboie]

What worked best for x86 was to move all the logic for translating the memory domain configuration to the actual memory memory management hardware from context switch time, to when the thread gets added to a memory domain, or the memory domain is modified. Each thread has its own set of page tables, so a context switch is now just a cr3 register update.

For an MPU system, what I'd like to see is the MPU register set mirrored in a data structure within the thread struct.

• These fields used to save/restore the MPU configuration when the thread is swapped in and out
• Modifications to the memory domain or a thread's membership to a domain, operate on this data structure synchronous with memory domain API calls, much like x86 now does.

[zephyrbot]

Hi @dcpleung,

This issue, marked as an Enhancement, was opened a while ago and did not get any traction. It was just assigned to you based on the labels. If you don't consider yourself the right person to address this issue, please re-assing it to the right person.

Please take a moment to review if the issue is still relevant to the project. If it is, please provide feedback and direction on how to move forward. If it is not, has already been addressed, is a duplicate, or is no longer relevant, please close it with a short comment explaining the reason.

@andrewboie you are also encouraged to help moving this issue forward by providing additional information and confirming this request/issue is still relevant to you.

Thanks!

[dcpleung]

Hm... this is highly implementation specific. So this will need to defer to maintainer of each arch.