deephacks / lmdbjni

LMDB for Java
Apache License 2.0
204 stars 28 forks source link

Windows version allocates entire database up front #15

Closed ccleve closed 9 years ago

ccleve commented 9 years ago

In the Linux version, a call to env.setMapSize() will create a small file that grows over time.

In the Windows version, said call will create a very large file immediately.

We need to do a bit of research to see if this is a Windows limitation or if there is a workaround.

krisskross commented 9 years ago

There is a discussion on this here http://ayende.com/blog/162787/reviewing-lightning-memory-mapped-database-library-because-damn-it.

The conclusion by Howard Chu (author of LMDB) is, "Windows will force the file to be fully allocated regardless. There's no point in trying to avoid it."

krisskross commented 9 years ago

Taken from CreateFileMapping function for future reference.

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366537%28v=vs.85%29.aspx

"If an application specifies a size for the file mapping object that is larger than the size of the actual named file on disk and if the page protection allows write access (that is, the flProtect parameter specifies PAGE_READWRITE or PAGE_EXECUTE_READWRITE), then the file on disk is increased to match the specified size of the file mapping object. If the file is extended, the contents of the file between the old end of the file and the new end of the file are not guaranteed to be zero; the behavior is defined by the file system. If the file on disk cannot be increased, CreateFileMapping fails and GetLastError returns ERROR_DISK_FULL."

krisskross commented 9 years ago

Ok to close?

ccleve commented 9 years ago

Yes, ok to close. I confirmed that Windows behaves this way. In Java, if you create a MappedByteBuffer it allocates the entire file instead of growing as needed.