search

microsoftarchive / redis

Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes
http://redis.io
Other
20.78k stars 5.37k forks source link

Redis commit size #529

Open awaizman1 opened 7 years ago

awaizman1 commented 7 years ago

Hi,

I'm running redis with no persistence (persistence-available no), when putting data of total size X I see (using VMMap/task manager) that the commit size for redis process is ~2X (working set is X).

I understand that the reason for 2X factor is that redis uses memory mapped heap in order to share memory with forked processes.

I'm not using persistence and this 2X becomes critical to me since I'm not working with page file on my system. That means that when I put into redis data of ~half of my RAM size, the commit size of redis process reaches the commit limit of the system and it crashes.

Is there some configuration I can apply to work over it (so that in no persistence mode it won't use memory mapped heap)?

Thanks, Assaf.

jepickett commented 7 years ago

The no persistence mode disables fork() emulation. In this case memory allocator is the Windows heap allocator. This memory allocator will do better with the page file turned on. If the page file is disabled, you will run into heap fragmentation issues that will effectively limit your use of redis to about one half of physical memory.

awaizman1 commented 7 years ago

Thanks,

I understand the implications of working without pagefile and going to change this. But the 2X commit size I see is on a fresh server and using only set operations on it, so fragmentation shouldn't be introduced yet.

I guess the 'Windows heap allocator' you mentioned is not the windows native heap, using the native heap would give X commit size. Does windows redis uses memory mapped file as its heap?