How to use the runtime paging plugin?

[AlfiRam]

I am trying to experiment with the Keystone runtime paging plugins to understand how the enclave self-paging works. However, after enabling the paging plugins in the runtime configuration, I encountered a "no backing store found" error.

I am aware that increasing the free memory size in the host file is a sensible solution (I have done this and it did work fine), but I specifically want to explore and utilize the runtime paging plugins to handle insufficient enclave memory.

I have enabled the paging plugins in the keystone/runtime/CMakeLists.txt configuration by setting the following options:

rt_option(PAGING "Enable runtime paging" ON)
rt_option(PAGE_CRYPTO "Enable page confidentiality" ON)
rt_option(PAGE_HASH "Enable page integrity" ON)

When I run the enclave with the paging plugins enabled, I get the following output:

# ./sealdemoEnclave.ke
Verifying archive integrity... MD5 checksums are OK. All good.
Uncompressing Keystone Enclave Package
[warn] no backing store found (paging.c:44)
[runtime] page fault at 0xed860 on 0x2040107f88 (scause: 0xf)

According to the Keystone paper, the enclave should be able to handle page faults by securely paging out its content to a backing store when the enclave memory is insufficient. However, it seems that the backing store is not properly configured, as indicated by the warning message [warn] no backing store found (paging.c:44).

I would appreciate any guidance on how to properly resolve the "no backing store found" error when using the paging plugins.

[grg-haas]

Hi @AlfiRam! Unfortunately, I have not worked much with the paging subsystem so I'm not quite sure what's going on here. Looking at the code where this error originates, it looks like it queries the SM with a call to a multimem plugin. Looking then here in the SM, it seems that multimem functionality is guarded behind an ifdef. Just for fun, give this a try:

1. For whatever platform you're using, navigate to sm/plat/<your platform>/config.mk
2. To the platform-cflags-y variable, add the text -DPLUGIN_ENABLE_MULTIMEM=1

Does that change anything?

[AlfiRam]

Hi @grg-haas , thanks for the suggestion. I tried adding the -DPLUGIN_ENABLE_MULTIMEM=1 flag to the platform-cflags-y variable in the sm/plat/generic/config.mk file, as I am running Keystone on QEMU and assumed that the "generic" platform is the correct one. However, after running my enclave application again, I still encountered the same "no backing store found" error. Is there any additional configuration or setup required for the backing store?

[grg-haas]

Hmmmm, besides this I'm not too sure what may be going on. Just to double check, did you rebuild your SM after changing these flags?

BUILDROOT_TARGET="keystone-sm-dirclean" make
make -j32

[AlfiRam]

yes I have :)

[grg-haas]

Sorry @AlfiRam , I'm out of ideas! If you find out what's causing this, please do let me know!

[AlfiRam]

Thank you for your response, @grg-haas. I appreciate your effort to help.

I am curious though, since there can only be so much memory in an enclave, I'm wondering about strategies for handling memory-intensive applications like ML. The Keystone paper mentions enclave page swapping to handle page faults (which comes as an Eyrie plugin), with encryption (AES-128) and integrity protection (like Merkle trees) for swapped pages. However, since I'm facing issues implementing this (as shown above), what is your usual approach on handling memory intensive eapps other than just increasing the freemem size in the host app?

[grg-haas]

Yeah, I usually just bump the freemem size. Sometimes I've had issues where I run out of contiguous physical memory on the Linux side (I believe the default is 16M) but you can increase by specifying something like cma=128M on the kernel command line. For the generic platform, you can do this by modifying the -append option in mkutils/plat/generic/run.mk and adding that option there.

[AlfiRam]

Thank you for your response @grg-haas . I believe I understand the purpose of Keystone's paging plugin better now.

From my understanding, Keystone's primary protection mechanism uses RISC-V PMP for enclaves, which is mainly designed to defend against software attacks, such as preventing the OS from accessing enclave content.

Additionally, from the Keystone paper, it seems the page encryption (AES) and integrity verification (Merkle tree) features are primarily designed to protect against physical attackers with access to DRAM. The paper demonstrates these features on the FU540-C000 hardware.

Given that I'm testing Keystone on QEMU, which is an emulator, I'm wondering if it's even possible to meaningfully implement or test these memory encryption and integrity tree features in this environment. Are these features only applicable when running on actual hardware like the FU540-C000?