search

dragonflydb / dragonfly

A modern replacement for Redis and Memcached
https://www.dragonflydb.io/
Other
25.96k stars 954 forks source link

debug populate doubles RSS #4124

Open kostasrim opened 1 week ago

kostasrim commented 1 week ago

Spoted here https://github.com/dragonflydb/dragonfly/pull/4117

Run: 

debug populate 1 prefix 1000000 TYPE hash RAND ELEMENTS 1000

Creates a hash table with 1GB total size. Now do INFO MEMORY. RSS is: 2062901248 (2GB ~ doubled).

adiholden commented 1 week ago

it looks like after running memory decommit after the debug populate command the memory drops to 1G, so it looks like it is a temporary allocation for generating the strings to insert

kostasrim commented 1 week ago

@adiholden shall we run it unconditionally after debug populate similarly to what we did with rdb?

adiholden commented 1 week ago

the problem relies in the fact that the batch in debug populate is on number of keys and not number of elements, we can fix this