#265 Update MemoryMappedPageManager memory mapped initiation

[ssonthal]

265

Two changes:

1. Creating the MemoryMappedFile without setting the total file size upfront

_mapped = MemoryMappedFile.CreateFromFile(_file, null, 0, MemoryMappedFileAccess.ReadWrite, HandleInheritability.None, true);

2. Instead of mapping the whole file, mapping only a portion of it dynamically as needed

_whole = _mapped.CreateViewAccessor(0, historyDepth * Page.PageSize);

[ssonthal]

Hi @Scooletz, pls let me know if these changes are in the right direction.

[ssonthal]

Provided comments. I don't understand how this could work without significant changes or using the NT method mentioned in the referred issue. Maybe we should add some tests for the MemoryMappedPageManager to ensure that it works properly? Also, add some matrix for OS dependent tests?

I read somewhere that the .NET memory-mapped file API abstracts the complexity of memory management, so we don’t need to use lower-level functions like NtCreateSection in C#.

Might be completely wrong here.

Do you suggest using ntdll.dll and create a custom class like this to get the same behavior as lmdb? -

public static class NativeMethods
{
    [DllImport("ntdll.dll")]
    public static extern NTSTATUS NtCreateSection(
        out IntPtr SectionHandle,
        ACCESS_MASK DesiredAccess,
        ref OBJECT_ATTRIBUTES ObjectAttributes,
        ref LARGE_INTEGER MaximumSize,
        uint SectionPageProtection,
        uint AllocationAttributes,
        IntPtr FileHandle);
}

Although since ntdll.dll is an unmanaged dll, on Linux machines, it will not work.

[Scooletz]

Although since ntdll.dll is an unmanaged dll, on Linux machines, it will not work.

The issue is pure Windows issue (see #265 that you linked). On Linux, mmap behaves nicely not preallocating whole file up front. The only method that I partially investigated was the mentioned NtCreateSection, but it might be the case that there are other ways. One way or the other, the crux is that on Windows it bloats to the file size defined at the beginning.

[Scooletz]

Any comments @shubham-sonthalia ?