search

redis / redis

Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.
http://redis.io
Other
66.36k stars 23.73k forks source link

high mem_fragmentation_ratio #7741

Open lw5885799 opened 4 years ago

lw5885799 commented 4 years ago

version: 5.0.2 problem: high mem_fragmentation_ratio

info memory: used_memory_human: 24.69M used_memory_rss_human: 236.4M mem_fragmentation_ratio: 9.59 mem_allocator: libc

I checked some article. Usually, the mem_fragmentation_ratio is between (1- 1.6). I have no idea why it happenned. how can I get log or other way to find the cause?

oranagra commented 4 years ago

@lw5885799 We currently have limited visibility into the internals of libc malloc (a lot more details when using jemalloc). Maybe we can figure something out by looking at /proc/<pid>/smaps

lw5885799 commented 4 years ago

@oranagra is there any reference to analyze the proc//smaps. I know few about this memory log. I mean how to find the cause of high mem_fragmentation_ratio by the smaps log.

oranagra commented 4 years ago

just see which mapping consumes the majority of RSS and if you can conclude that's the libc allocator heap or not.

maybe you'll be able to prove that it's related to some other library that got loaded, or maybe that's a memory mapped file.

if that's the allocator then you need to find a way to look into it's internals. for jemalloc we have je_malloc_stats_print which exposes a lot of data. not sure what is the equivalent for libc malloc, but if you find one, feel free to make a PR for the MEMORY MALLOC-STATS command..

btw, why not use jemalloc?

lw5885799 commented 4 years ago

Thanks. I will try this. That guy who building the cluster accidently changed the allocator in product environment. We have changed allocator to jemalloc. Still, want to know what happened

lw5885799 commented 4 years ago

I use the memory doctor command in a 4.65 mem_fragmentation_ratio cluster. It says after a memory peak, mem_fragmentation_ratio would be high. Also, it says it is a normal and harmless isuess. As long as I fill it up, mem_fragmentation_ratio goes down. It happened with a jemalloc allocator. I wonder if malloc would lead to the same thing?

My old cluster with malloc had a used_memory_rss 5.5GB used_memory_peak 4.9GB and used_memory 39MB. The mem_fragmentation_ratio reached to 143. Is it still a normal thing?

oranagra commented 4 years ago

if the rss matches the peak memory, it could just be fragmentation, or some issue with the allcator. but the numbers seems very unlikely. if we allocated 5gb, and then released 90% of that memory, i assume there must be some pages that the allocator can return to the OS. in other words, it seems unlikely that it'll keep so much memory as rss.

it's hard to tell for sure because we don't have a lot of info with libc allocator. p.s. when using jemalloc, redis has an activedefrag feature that can solve this.