lf-edge / eve

EVE is Edge Virtualization Engine
https://www.lfedge.org/projects/eve/
Apache License 2.0
475 stars 162 forks source link

Effects of enabling Kernel SamePage Merging (KSM) by default on EVE #4250

Closed roja-zededa closed 1 month ago

roja-zededa commented 1 month ago

Feature

The Linux kernel uses KSM to reduce memory usage of VMs by merging duplicated pages. What are the implications of enabling KSM on EVE?

Possible effects of enabling KSM on EVE

rouming commented 1 month ago

Able to launch more VMs without waking up OOM killer

I would say this is the opposite: enabling KSM would lead to memory over-provisioning and potential risk of more frequent OOM kills. Reasoning is simple: imagine N vms share M physical pages which tightly fit the host RAM. Imagine what would happen if one of the vms starts writing to these shared pages: bunch of copy-on-write page faults lead to more pages allocations, which are not available on the host due to the over-provisioned memory. Game over.

Sharing of RO only pages would make more sense, but I'm not sure we can easily get RO regions from guest RAM (probably a "hint" from the guest driver).

The other concern is performance: KSM merging thread which does memcmp in the background is tolerable (can be executed on a dedicated CPU, not shared with guests), but bunch of COW page faults can randomly slow down guest memory write operations.

In EVE we have a policy of NOT supporting memory over-provisioning (and storage as well btw), which makes all optimizations not interesting and impractical: why to bother if we already give everything to a vm what it wants.