search

StephanvanSchaik / mmap-rs

A cross-platform and safe Rust API to create and manage memory mappings in the virtual address space of the calling process.
Apache License 2.0
59 stars 17 forks source link

Support DONT_COMMIT to reserve (overcommit) on Windows #12

Closed s1341 closed 1 year ago

s1341 commented 1 year ago

This PR adds support for a DONT_COMMIT unsafe flag, which allows for reserved but uncommitted allocations on windows. This is known as 'overcommit' on unix, and is the default there.

StephanvanSchaik commented 1 year ago

Hi!

Thank you for your PR!

Since it is pretty common to use PROT_NONE to reserve memory regions on Unix, I feel that it would be nicer to just not commit the pages when calling VirtualAlloc with PAGE_NOACCESS. Since using MmapOptions::map_none() should result in a memory mapping that is not accessible anyway, this should be possible without having to rely on unsafe Rust (e.g. UnsafeFlags). In addition, this would result in a consistent interface between platforms to reserve regions in the virtual address space without committing any memory.

I went ahead and implemented this in PR #13. Can you let me know if this works for you?

s1341 commented 1 year ago

I'll test it. Although on unix you can allocate with e.g. PROT_READ|PROT_WRITE and have it overcommit (i.e. reserve but not commit pages). You then do not need to mprotect the memory before using it. It is commited as used. I guess doing it your way provides a more consistent usage experience...

StephanvanSchaik commented 1 year ago

I'll test it. Although on unix you can allocate with e.g. PROT_READ|PROT_WRITE and have it overcommit (i.e. reserve but not commit pages). You then do not need to mprotect the memory before using it. It is commited as used. I guess doing it your way provides a more consistent usage experience...

PR #27 introduces Reserved, ReservedNone and ReservedMut. So rather than using MmapOptions::map(), you can now use MmapOptions::reserve() to reserve immutable memory (i.e. memory with read permissions), and then use Reserved::try_into() to convert it into a Mmap object, thus committing the memory mapping on Microsoft Windows. This also avoids the need to call mprotect() on Unix platforms and VirtualProtect() on Microsoft Windows.

This PR also includes a number of tests to ensure this all works as intended, since PR #26 fixes another issue related to reserving memory on Microsoft Windows. More specifically, in the cases that Mmap::split_off() and Mmap::split_to() are being used to split the Mmap object into multiple such objects (see also PR #18).

As I think this API adequately addresses this issue, I am closing this PR and PR #13. Feel free to open an issue if this is not the case, though.