Closed GoogleCodeExporter closed 8 years ago
I have a custom built tool called "redis-top" that show me some info of my
redis cluster.
Before the "leak":
INSTANCE PING db0:keys mem mem_real clients
reqs/s role
10.1.2.201:63700 0.188ms 208949 54.03M 64.56M 37
20 master
10.1.2.201:63701 0.143ms 209183 54.12M 61.21M 37
15 master
10.1.2.201:63702 0.144ms 208997 53.95M 60.99M 37
15 master
10.1.2.201:63703 0.136ms 209024 54.08M 64.66M 37
14 master
10.1.2.201:63704 0.140ms 208770 53.95M 60.99M 38
13 master
10.1.2.201:63710 0.138ms 208084 53.86M 60.88M 37
20 master
10.1.2.201:63711 0.141ms 208589 53.97M 61.08M 38
14 slave
10.1.2.201:63712 0.138ms 208369 53.96M 61.07M 38
20 slave
10.1.2.201:63713 0.142ms 208297 53.91M 61.02M 38
13 slave
10.1.2.201:63714 0.141ms 208285 53.91M 61.01M 39
15 slave
10.1.2.202:63700 0.131ms 209128 54.04M 61.09M 37
15 master
10.1.2.202:63701 0.111ms 208146 53.83M 60.85M 37
20 master
10.1.2.202:63702 0.135ms 208973 54.04M 61.09M 37
14 master
10.1.2.202:63703 0.134ms 209071 54.12M 61.16M 37
17 master
10.1.2.202:63704 0.136ms 209366 54.18M 61.23M 37
19 master
10.1.2.202:63710 0.129ms 208949 54.04M 61.15M 38
22 slave
10.1.2.202:63711 0.134ms 209183 54.13M 61.27M 38
12 slave
10.1.2.202:63712 0.157ms 208997 53.96M 61.07M 38
18 slave
10.1.2.202:63713 0.125ms 209024 54.09M 61.22M 38
12 slave
10.1.2.202:63714 0.141ms 208770 53.95M 61.07M 38
14 slave
10.1.2.203:63700 0.118ms 208271 53.89M 61.12M 37
16 master
10.1.2.203:63701 0.115ms 209339 54.25M 64.88M 37
15 master
10.1.2.203:63702 0.116ms 209128 54.12M 61.16M 37
14 master
10.1.2.203:63703 0.130ms 208260 53.88M 60.88M 37
13 master
10.1.2.203:63704 0.117ms 208562 54.03M 64.57M 39
16 master
10.1.2.203:63710 0.139ms 209128 54.04M 61.18M 38
17 slave
10.1.2.203:63711 0.117ms 208146 53.85M 60.96M 39
18 slave
10.1.2.203:63712 0.115ms 208973 54.05M 61.16M 38
16 slave
10.1.2.203:63713 0.112ms 209071 54.13M 61.25M 39
20 slave
10.1.2.203:63714 0.115ms 209366 54.19M 61.32M 38
18 slave
10.1.2.204:63700 0.141ms 208615 53.96M 64.43M 37
12 master
10.1.2.204:63701 0.134ms 209099 54.01M 61.05M 37
15 master
10.1.2.204:63702 0.118ms 208322 53.92M 60.96M 37
18 master
10.1.2.204:63703 0.131ms 210035 54.31M 61.39M 37
15 master
10.1.2.204:63704 0.116ms 208871 54.01M 61.06M 37
12 master
10.1.2.204:63710 0.137ms 208271 53.89M 60.98M 38
13 slave
10.1.2.204:63711 0.117ms 209339 54.25M 61.39M 38
15 slave
10.1.2.204:63712 0.131ms 209128 54.13M 61.27M 38
16 slave
10.1.2.204:63713 0.116ms 208260 53.88M 61.00M 38
12 slave
10.1.2.204:63714 0.153ms 208562 54.02M 61.13M 38
17 slave
10.1.2.205:63700 0.116ms 209078 54.07M 61.14M 37
13 master
10.1.2.205:63701 0.114ms 209411 54.13M 61.18M 38
15 master
10.1.2.205:63702 0.118ms 208898 54.00M 64.54M 37
15 master
10.1.2.205:63703 0.115ms 208243 53.82M 64.34M 37
16 master
10.1.2.205:63704 0.115ms 208580 53.98M 64.49M 37
17 master
10.1.2.205:63710 0.116ms 208615 53.96M 61.07M 38
15 slave
10.1.2.205:63711 0.117ms 209099 54.02M 61.16M 38
11 slave
10.1.2.205:63712 0.115ms 208322 53.94M 61.06M 40
19 slave
10.1.2.205:63713 0.130ms 210035 54.32M 61.48M 38
16 slave
10.1.2.205:63714 0.115ms 208871 54.02M 61.16M 38
16 slave
10.1.2.206:63700 0.141ms 207954 53.79M 60.81M 38
17 master
10.1.2.206:63701 0.136ms 209282 54.21M 61.31M 37
13 master
10.1.2.206:63702 0.136ms 208520 53.88M 64.41M 37
13 master
10.1.2.206:63703 0.136ms 208758 54.00M 64.54M 37
18 master
10.1.2.206:63704 0.139ms 208926 54.07M 64.59M 37
19 master
10.1.2.206:63710 0.115ms 209078 54.08M 61.20M 38
12 slave
10.1.2.206:63711 0.134ms 209411 54.13M 61.28M 38
15 slave
10.1.2.206:63712 0.138ms 208898 54.01M 61.16M 38
13 slave
10.1.2.206:63713 0.138ms 208243 53.83M 60.95M 39
17 slave
10.1.2.206:63714 0.136ms 208580 53.98M 61.12M 38
16 slave
10.1.2.207:63700 0.102ms 208555 53.98M 61.27M 38
15 master
10.1.2.207:63701 0.063ms 208060 53.85M 60.89M 37
17 master
10.1.2.207:63702 0.062ms 209535 54.22M 64.79M 37
18 master
10.1.2.207:63703 0.061ms 209087 53.98M 61.05M 37
15 master
10.1.2.207:63704 0.066ms 209219 54.00M 64.52M 37
15 master
10.1.2.207:63710 0.062ms 207954 53.79M 60.89M 38
15 slave
10.1.2.207:63711 0.061ms 209282 54.22M 61.34M 38
14 slave
10.1.2.207:63712 0.063ms 208520 53.89M 60.98M 38
17 slave
10.1.2.207:63713 0.084ms 208758 54.00M 61.12M 38
19 slave
10.1.2.207:63714 0.085ms 208926 54.07M 61.18M 38
17 slave
10.1.2.208:63700 0.181ms 208715 53.93M 64.64M 38
16 master
10.1.2.208:63701 0.135ms 208707 54.03M 61.07M 38
16 master
10.1.2.208:63702 0.127ms 209265 54.17M 61.23M 37
16 master
10.1.2.208:63703 0.130ms 208574 53.97M 61.03M 37
18 master
10.1.2.208:63704 0.130ms 209065 54.04M 61.09M 37
15 master
10.1.2.208:63710 0.116ms 208555 53.98M 61.09M 38
17 slave
10.1.2.208:63711 0.117ms 208060 53.87M 60.96M 39
15 slave
10.1.2.208:63712 0.129ms 209535 54.23M 61.37M 39
18 slave
10.1.2.208:63713 0.115ms 209087 53.98M 61.11M 38
17 slave
10.1.2.208:63714 0.134ms 209219 54.01M 61.13M 38
19 slave
10.1.2.209:63700 0.154ms 209314 54.12M 64.69M 37
17 master
10.1.2.209:63701 0.133ms 208102 53.88M 64.39M 37
18 master
10.1.2.209:63702 0.139ms 208628 53.95M 64.46M 37
13 master
10.1.2.209:63703 0.143ms 208303 53.96M 64.48M 37
16 master
10.1.2.209:63704 0.135ms 208321 53.87M 64.32M 37
14 master
10.1.2.209:63710 0.129ms 208715 53.92M 61.04M 38
14 slave
10.1.2.209:63711 0.133ms 208707 54.03M 61.13M 38
14 slave
10.1.2.209:63712 0.143ms 209265 54.18M 61.29M 38
12 slave
10.1.2.209:63713 0.133ms 208574 53.98M 61.09M 38
20 slave
10.1.2.209:63714 0.133ms 209065 54.05M 61.16M 39
15 slave
10.1.2.210:63700 0.207ms 207991 53.76M 60.79M 37
16 master
10.1.2.210:63701 0.136ms 209199 54.12M 61.18M 37
14 master
10.1.2.210:63702 0.136ms 209260 54.12M 61.17M 37
12 master
10.1.2.210:63703 0.138ms 209156 54.08M 61.13M 38
19 master
10.1.2.210:63704 0.136ms 209348 54.10M 61.15M 37
13 master
10.1.2.210:63705 0.136ms 209205 54.05M 61.11M 37
11 master
10.1.2.210:63706 0.145ms 208738 53.94M 60.95M 37
16 master
10.1.2.210:63710 0.115ms 209314 54.12M 61.25M 38
15 slave
10.1.2.210:63711 0.142ms 208102 53.89M 60.97M 38
17 slave
10.1.2.210:63712 0.114ms 208628 53.96M 61.07M 38
14 slave
10.1.2.210:63713 0.136ms 208303 53.96M 61.06M 38
18 slave
10.1.2.210:63714 0.134ms 208321 53.88M 60.97M 39
18 slave
10.1.2.211:63700 0.115ms 208863 54.12M 61.17M 37
16 master
10.1.2.211:63701 0.114ms 208752 53.98M 64.53M 37
16 master
10.1.2.211:63702 0.119ms 209382 54.17M 64.72M 37
16 master
10.1.2.211:63703 0.115ms 209146 54.04M 64.53M 37
16 master
10.1.2.211:63704 0.115ms 208578 53.93M 64.50M 37
18 master
10.1.2.211:63705 0.123ms 208672 53.99M 64.58M 37
14 master
10.1.2.211:63706 0.115ms 208115 53.89M 64.40M 37
12 master
10.1.2.211:63710 0.111ms 207991 53.77M 60.87M 38
12 slave
10.1.2.211:63711 0.121ms 209199 54.13M 61.26M 38
17 slave
10.1.2.211:63712 0.115ms 209260 54.13M 61.25M 39
12 slave
10.1.2.211:63713 0.115ms 209156 54.08M 61.20M 38
16 slave
10.1.2.211:63714 0.117ms 209348 54.10M 61.23M 38
13 slave
10.1.2.211:63715 0.115ms 209205 54.06M 61.18M 39
14 slave
10.1.2.211:63716 0.115ms 208738 53.94M 61.05M 38
18 slave
10.1.2.212:63700 0.117ms 208084 53.87M 61.02M 38
12 slave
10.1.2.212:63701 0.117ms 208589 53.96M 61.00M 37
19 master
10.1.2.212:63702 0.116ms 208369 53.96M 64.47M 37
17 master
10.1.2.212:63703 0.118ms 208297 53.90M 64.40M 37
10 master
10.1.2.212:63704 0.116ms 208285 53.89M 64.47M 37
16 master
10.1.2.212:63710 0.115ms 208863 54.13M 61.25M 38
15 slave
10.1.2.212:63711 0.118ms 208752 53.99M 61.11M 38
16 slave
10.1.2.212:63712 0.114ms 209382 54.17M 61.32M 38
14 slave
10.1.2.212:63713 0.116ms 209146 54.05M 61.18M 39
17 slave
10.1.2.212:63714 0.115ms 208578 53.94M 61.03M 38
17 slave
10.1.2.212:63715 0.113ms 208672 54.00M 61.11M 39
13 slave
10.1.2.212:63716 0.114ms 208115 53.91M 60.99M 39
15 slave
AVERAGE: 0.124ms 208800 54.01M 61.86M 37
15 -
TOTAL: 15.879ms 26726476 6.75G 7.73G 4824
1992 master[64]/slave[64]
Original comment by daniel.m...@gmail.com
on 15 Jun 2011 at 9:12
After the "leak":
INSTANCE PING db0:keys mem mem_real clients reqs/s
role
10.1.2.201:63700 0.182ms 208921 54.03M 235.95M 37 27
master
10.1.2.201:63701 0.139ms 209155 54.13M 205.77M 37 16
master
10.1.2.201:63702 0.133ms 208970 53.95M 233.67M 37 18
master
10.1.2.201:63703 0.137ms 209016 54.09M 227.52M 37 16
master
10.1.2.201:63704 0.133ms 208751 53.94M 235.67M 37 21
master
10.1.2.201:63710 0.136ms 208030 53.86M 253.34M 38 18
slave
10.1.2.201:63711 0.134ms 208534 53.96M 281.63M 38 20
slave
10.1.2.201:63712 0.135ms 208327 53.97M 254.35M 38 22
slave
10.1.2.201:63713 0.134ms 208259 53.91M 254.46M 38 19
slave
10.1.2.201:63714 0.134ms 208230 53.90M 259.59M 38 19
slave
10.1.2.202:63700 0.177ms 209075 54.03M 229.72M 37 17
master
10.1.2.202:63701 0.131ms 208073 53.82M 185.42M 37 28
master
10.1.2.202:63702 0.131ms 208928 54.04M 237.16M 37 16
master
10.1.2.202:63703 0.129ms 209026 54.12M 234.61M 38 15
master
10.1.2.202:63704 0.135ms 209317 54.17M 179.53M 37 15
master
10.1.2.202:63710 0.165ms 208921 54.05M 233.37M 39 29
slave
10.1.2.202:63711 0.134ms 209155 54.13M 226.41M 38 16
slave
10.1.2.202:63712 0.128ms 208970 53.96M 232.53M 38 20
slave
10.1.2.202:63713 0.130ms 209016 54.10M 229.45M 38 18
slave
10.1.2.202:63714 0.130ms 208751 53.95M 230.12M 38 16
slave
10.1.2.203:63700 0.173ms 208196 53.87M 266.04M 37 16
master
10.1.2.203:63701 0.108ms 209285 54.24M 252.48M 37 18
master
10.1.2.203:63702 0.108ms 209047 54.11M 251.21M 37 13
master
10.1.2.203:63703 0.108ms 208195 53.87M 253.29M 37 19
master
10.1.2.203:63704 0.127ms 208501 54.01M 274.39M 38 14
master
10.1.2.203:63710 0.127ms 209075 54.04M 198.55M 39 16
slave
10.1.2.203:63711 0.108ms 208073 53.82M 182.88M 38 22
slave
10.1.2.203:63712 0.127ms 208928 54.05M 181.68M 38 15
slave
10.1.2.203:63713 0.126ms 209026 54.12M 199.93M 38 18
slave
10.1.2.203:63714 0.110ms 209317 54.18M 182.34M 38 16
slave
10.1.2.204:63700 0.110ms 208554 53.95M 232.17M 37 16
master
10.1.2.204:63701 0.110ms 209045 54.01M 208.48M 38 14
master
10.1.2.204:63702 0.119ms 208280 53.93M 238.91M 38 32
master
10.1.2.204:63703 0.110ms 209972 54.30M 225.23M 37 24
master
10.1.2.204:63704 0.119ms 208801 54.00M 241.37M 37 15
master
10.1.2.204:63710 0.115ms 208196 53.87M 252.14M 38 20
slave
10.1.2.204:63711 0.109ms 209285 54.24M 250.15M 38 14
slave
10.1.2.204:63712 0.109ms 209047 54.11M 252.90M 38 13
slave
10.1.2.204:63713 0.112ms 208195 53.87M 243.80M 38 14
slave
10.1.2.204:63714 0.109ms 208501 54.01M 264.73M 38 19
slave
10.1.2.205:63700 0.129ms 209048 54.07M 237.50M 37 23
master
10.1.2.205:63701 0.129ms 209379 54.13M 241.97M 37 16
master
10.1.2.205:63702 0.108ms 208856 54.00M 232.32M 37 15
master
10.1.2.205:63703 0.129ms 208209 53.82M 245.21M 38 13
master
10.1.2.205:63704 0.129ms 208540 53.98M 238.80M 38 15
master
10.1.2.205:63710 0.129ms 208554 53.95M 231.04M 38 21
slave
10.1.2.205:63711 0.107ms 209045 54.01M 235.06M 38 16
slave
10.1.2.205:63712 0.125ms 208280 53.92M 220.96M 38 35
slave
10.1.2.205:63713 0.129ms 209972 54.31M 240.30M 38 23
slave
10.1.2.205:63714 0.128ms 208801 54.01M 238.94M 38 17
slave
10.1.2.206:63700 0.165ms 207893 53.78M 181.81M 38 17
master
10.1.2.206:63701 0.130ms 209228 54.21M 171.57M 37 19
master
10.1.2.206:63702 0.129ms 208459 53.87M 200.45M 37 16
master
10.1.2.206:63703 0.130ms 208667 53.99M 167.21M 38 18
master
10.1.2.206:63704 0.130ms 208870 54.07M 236.81M 38 20
master
10.1.2.206:63710 0.135ms 209048 54.08M 210.68M 39 19
slave
10.1.2.206:63711 0.136ms 209379 54.13M 235.06M 38 21
slave
10.1.2.206:63712 0.129ms 208856 54.01M 216.14M 38 15
slave
10.1.2.206:63713 0.130ms 208209 53.83M 232.78M 39 20
slave
10.1.2.206:63714 0.130ms 208540 53.98M 234.30M 38 14
slave
10.1.2.207:63700 0.136ms 208534 53.98M 221.18M 37 15
master
10.1.2.207:63701 0.076ms 208019 53.86M 229.02M 38 18
master
10.1.2.207:63702 0.055ms 209484 54.22M 221.63M 37 14
master
10.1.2.207:63703 0.056ms 209032 53.97M 222.16M 37 16
master
10.1.2.207:63704 0.052ms 209176 54.00M 240.24M 37 21
master
10.1.2.207:63710 0.104ms 207893 53.78M 181.30M 38 17
slave
10.1.2.207:63711 0.060ms 209228 54.21M 198.84M 38 16
slave
10.1.2.207:63712 0.063ms 208459 53.88M 197.96M 38 15
slave
10.1.2.207:63713 0.055ms 208667 53.99M 175.25M 38 15
slave
10.1.2.207:63714 0.055ms 208870 54.07M 188.54M 38 17
slave
10.1.2.208:63700 0.143ms 208680 53.92M 215.43M 37 17
master
10.1.2.208:63701 0.142ms 208677 54.03M 221.60M 38 19
master
10.1.2.208:63702 0.109ms 209197 54.16M 230.70M 38 20
master
10.1.2.208:63703 0.108ms 208505 53.97M 230.25M 38 18
master
10.1.2.208:63704 0.110ms 209021 54.03M 241.14M 37 20
master
10.1.2.208:63710 0.126ms 208534 53.99M 220.72M 39 14
slave
10.1.2.208:63711 0.110ms 208019 53.86M 224.62M 38 18
slave
10.1.2.208:63712 0.109ms 209484 54.23M 222.55M 38 16
slave
10.1.2.208:63713 0.123ms 209032 53.99M 216.52M 39 17
slave
10.1.2.208:63714 0.126ms 209176 54.00M 222.39M 38 22
slave
10.1.2.209:63700 0.172ms 209258 54.11M 250.35M 37 20
master
10.1.2.209:63701 0.134ms 208029 53.88M 254.50M 39 35
master
10.1.2.209:63702 0.134ms 208590 53.96M 250.89M 38 17
master
10.1.2.209:63703 0.130ms 208238 53.94M 243.12M 37 18
master
10.1.2.209:63704 0.131ms 208270 53.87M 229.43M 38 13
master
10.1.2.209:63710 0.133ms 208680 53.93M 218.01M 39 17
slave
10.1.2.209:63711 0.138ms 208677 54.03M 218.97M 38 14
slave
10.1.2.209:63712 0.130ms 209197 54.16M 227.55M 38 19
slave
10.1.2.209:63713 0.132ms 208505 53.97M 220.27M 38 13
slave
10.1.2.209:63714 0.132ms 209021 54.05M 219.36M 39 18
slave
10.1.2.210:63700 0.108ms 207878 53.74M 255.69M 38 15
master
10.1.2.210:63701 0.108ms 209065 54.09M 331.42M 37 17
master
10.1.2.210:63702 0.109ms 209139 54.08M 255.67M 37 18
master
10.1.2.210:63703 0.126ms 209023 54.03M 254.51M 37 15
master
10.1.2.210:63704 0.108ms 209224 54.08M 326.20M 38 18
master
10.1.2.210:63705 0.122ms 209079 54.02M 279.91M 37 11
master
10.1.2.210:63706 0.123ms 208619 53.92M 254.12M 39 15
master
10.1.2.210:63710 0.106ms 209258 54.12M 231.06M 38 20
slave
10.1.2.210:63711 0.120ms 208029 53.87M 250.35M 38 36
slave
10.1.2.210:63712 0.181ms 208590 53.96M 246.51M 38 13
slave
10.1.2.210:63713 0.107ms 208238 53.95M 245.25M 38 17
slave
10.1.2.210:63714 0.126ms 208270 53.88M 228.40M 39 16
slave
10.1.2.211:63700 0.125ms 208805 54.12M 249.58M 38 22
master
10.1.2.211:63701 0.109ms 208707 53.98M 250.25M 37 15
master
10.1.2.211:63702 0.128ms 209337 54.17M 327.93M 38 26
master
10.1.2.211:63703 0.130ms 209074 54.02M 252.71M 37 15
master
10.1.2.211:63704 0.107ms 208515 53.93M 250.50M 38 13
master
10.1.2.211:63705 0.106ms 208626 53.98M 248.28M 37 20
master
10.1.2.211:63706 0.106ms 208073 53.89M 242.56M 37 18
master
10.1.2.211:63710 0.106ms 207878 53.74M 277.07M 39 18
slave
10.1.2.211:63711 0.118ms 209065 54.09M 256.86M 38 17
slave
10.1.2.211:63712 0.158ms 209139 54.10M 254.02M 39 14
slave
10.1.2.211:63713 0.108ms 209023 54.05M 249.58M 39 14
slave
10.1.2.211:63714 0.108ms 209224 54.07M 252.80M 38 17
slave
10.1.2.211:63715 0.097ms 209079 54.02M 252.08M 38 14
slave
10.1.2.211:63716 0.109ms 208619 53.91M 250.79M 38 19
slave
10.1.2.212:63700 0.109ms 208030 53.86M 281.98M 37 17
master
10.1.2.212:63701 0.111ms 208534 53.95M 251.40M 37 20
master
10.1.2.212:63702 0.122ms 208327 53.96M 252.13M 37 23
master
10.1.2.212:63703 0.106ms 208259 53.90M 254.01M 37 18
master
10.1.2.212:63704 0.108ms 208230 53.89M 254.90M 37 20
master
10.1.2.212:63710 0.106ms 208805 54.12M 277.70M 38 20
slave
10.1.2.212:63711 0.108ms 208707 53.98M 251.38M 38 15
slave
10.1.2.212:63712 0.104ms 209337 54.17M 279.05M 38 28
slave
10.1.2.212:63713 0.107ms 209074 54.03M 253.14M 38 16
slave
10.1.2.212:63714 0.108ms 208515 53.93M 248.76M 38 18
slave
10.1.2.212:63715 0.105ms 208626 54.00M 272.80M 39 15
slave
10.1.2.212:63716 0.105ms 208073 53.90M 247.99M 39 19
slave
AVERAGE: 0.120ms 208742 54.00M 236.84M 37 18
-
TOTAL: 15.359ms 26719022 6.75G 29.61G 4838 2318
master[64]/slave[64]
Original comment by daniel.m...@gmail.com
on 15 Jun 2011 at 9:12
It smells a lot like your MONITOR client isn't reading fast enough or your
network connection isn't fast enough. To verify, try running "redis-cli MONITOR
> /dev/null".
Original comment by josiah.c...@gmail.com
on 16 Jun 2011 at 12:40
This sounds like the same bug I reported in 525.
http://code.google.com/p/redis/issues/detail?id=525
Redis doesn't limit the amount of memory allocated for client buffers, which
would make sense that I see it on slow pub/sub clients and you see it with
monitor (a kind of subscription).
Original comment by jwillp@gmail.com
on 16 Jun 2011 at 12:44
Yeah, my client may not be reading fast enough, but that does mean that Redis
should allocate a large amount of memory and never free it afterward...
Original comment by daniel.m...@gmail.com
on 16 Jun 2011 at 1:29
This is caused by memory fragmentation. Reclaiming memory after the top of the
heap is grown is very difficult, as new allocations may have been done there.
Redis itself can't do anything about it, but using a different allocator can
decrease fragmentation drastically. We've seen good reports of people using
jemalloc. There is a separate branch available that includes jemalloc support:
https://github.com/antirez/redis/tree/2.2-jemalloc-static. It is not 100% up to
date with the latest 2.2, but good enough (unless you can tell from the
changelog that you need the latest 2.2, in that case I can update that branch
for you). Can you report back here if Redis with jemalloc has better results in
your scenario? Thanks.
Original comment by pcnoordh...@gmail.com
on 16 Jun 2011 at 1:56
Using the jemalloc branch:
INSTANCE PING db0:keys mem mem_real clients
reqs/s role
10.1.2.201:63700 0.182ms 208769 256.53M 228.65M 41
22 master
10.1.2.201:63701 0.144ms 208959 257.31M 228.88M 41
15 master
10.1.2.201:63702 0.176ms 208831 256.19M 226.74M 42
12 master
10.1.2.201:63703 0.140ms 208767 256.76M 234.09M 40
14 master
10.1.2.201:63704 0.138ms 208590 256.23M 227.94M 41
16 master
10.1.2.201:63710 0.143ms 207943 218.26M 192.63M 40
16 slave
10.1.2.201:63711 0.145ms 208386 217.91M 205.88M 40
14 slave
10.1.2.201:63712 0.123ms 208219 218.37M 191.70M 43
18 slave
10.1.2.201:63713 0.140ms 208123 218.18M 193.78M 41
13 slave
10.1.2.201:63714 0.147ms 208095 218.06M 194.16M 40
15 slave
10.1.2.202:63700 0.140ms 208920 257.21M 231.10M 41
14 master
10.1.2.202:63701 0.137ms 207886 256.05M 227.17M 41
17 master
10.1.2.202:63702 0.141ms 208815 256.91M 228.70M 41
16 master
10.1.2.202:63703 0.132ms 208891 257.10M 225.17M 42
13 master
10.1.2.202:63704 0.141ms 209182 258.14M 229.68M 39
15 master
10.1.2.202:63710 0.134ms 208769 219.70M 194.41M 44
20 slave
10.1.2.202:63711 0.134ms 208959 219.02M 195.18M 41
11 slave
10.1.2.202:63712 0.128ms 208831 218.85M 191.28M 42
16 slave
10.1.2.202:63713 0.139ms 208767 218.98M 201.70M 43
16 slave
10.1.2.202:63714 0.141ms 208590 218.41M 193.52M 42
14 slave
10.1.2.203:63700 0.135ms 208014 218.13M 188.89M 42
14 slave
10.1.2.203:63701 0.123ms 209169 220.61M 194.12M 40
15 slave
10.1.2.203:63702 0.104ms 208939 219.23M 190.57M 44
19 slave
10.1.2.203:63703 0.125ms 208072 218.51M 191.22M 41
14 slave
10.1.2.203:63704 0.124ms 208414 217.82M 194.94M 40
15 slave
10.1.2.203:63710 0.123ms 208920 218.44M 198.80M 41
15 slave
10.1.2.203:63711 0.125ms 207886 217.50M 192.65M 42
23 slave
10.1.2.203:63712 0.127ms 208815 218.93M 194.02M 41
13 slave
10.1.2.203:63713 0.132ms 208891 219.19M 194.10M 43
11 slave
10.1.2.203:63714 0.130ms 209182 219.21M 194.75M 43
13 slave
10.1.2.204:63700 0.129ms 208450 218.23M 196.43M 41
14 slave
10.1.2.204:63701 0.131ms 208960 218.76M 195.88M 42
13 slave
10.1.2.204:63702 0.121ms 208211 218.71M 195.11M 43
19 slave
10.1.2.204:63703 0.103ms 209791 258.13M 227.56M 42
13 master
10.1.2.204:63704 0.124ms 208643 218.75M 192.89M 42
11 slave
10.1.2.204:63710 0.123ms 208014 255.88M 224.86M 40
15 master
10.1.2.204:63711 0.125ms 209169 257.53M 225.18M 43
16 master
10.1.2.204:63712 0.123ms 208939 257.19M 226.00M 41
19 master
10.1.2.204:63713 0.127ms 208072 255.37M 222.70M 41
13 master
10.1.2.204:63714 0.129ms 208414 256.64M 227.93M 42
14 master
10.1.2.205:63700 0.151ms 208903 219.09M 196.53M 42
16 slave
10.1.2.205:63701 0.173ms 209233 219.32M 195.23M 42
13 slave
10.1.2.205:63702 0.149ms 208794 218.79M 194.59M 41
16 slave
10.1.2.205:63703 0.316ms 208086 217.72M 195.69M 41
15 slave
10.1.2.205:63704 0.152ms 208391 217.91M 205.69M 40
16 slave
10.1.2.205:63710 0.154ms 208450 255.35M 228.25M 40
12 master
10.1.2.205:63711 0.145ms 208960 256.13M 227.52M 42
14 master
10.1.2.205:63712 0.159ms 208211 255.59M 226.98M 44
17 master
10.1.2.205:63713 0.140ms 209791 219.72M 195.15M 41
16 slave
10.1.2.205:63714 0.150ms 208643 256.57M 224.08M 41
12 master
10.1.2.206:63700 0.147ms 207725 256.21M 225.64M 32
10 master
10.1.2.206:63701 0.140ms 208958 256.56M 228.36M 39
9 master
10.1.2.206:63702 0.141ms 208266 256.04M 226.17M 39
11 master
10.1.2.206:63703 0.899ms 208515 256.55M 224.41M 39
12 master
10.1.2.206:63704 0.637ms 208674 256.82M 228.78M 40
16 master
10.1.2.206:63710 0.208ms 208903 256.67M 228.51M 39
16 master
10.1.2.206:63711 0.199ms 209233 256.87M 226.51M 41
17 master
10.1.2.206:63712 0.222ms 208794 257.05M 224.91M 35
9 master
10.1.2.206:63713 0.261ms 208086 255.77M 227.80M 39
14 master
10.1.2.206:63714 0.141ms 208391 256.61M 236.82M 39
15 master
10.1.2.207:63700 0.111ms 208448 219.07M 197.62M 42
17 slave
10.1.2.207:63701 0.077ms 207825 217.59M 200.59M 41
15 slave
10.1.2.207:63702 0.067ms 209181 219.37M 197.64M 40
16 slave
10.1.2.207:63703 0.070ms 208890 218.27M 201.18M 40
13 slave
10.1.2.207:63704 0.061ms 208991 218.85M 196.86M 42
22 slave
10.1.2.207:63710 0.072ms 207725 217.88M 196.32M 42
15 slave
10.1.2.207:63711 0.069ms 208958 219.09M 196.71M 40
14 slave
10.1.2.207:63712 0.071ms 208266 218.65M 196.82M 41
17 slave
10.1.2.207:63713 0.074ms 208515 218.64M 194.20M 41
14 slave
10.1.2.207:63714 0.073ms 208674 218.78M 200.93M 44
11 slave
10.1.2.208:63700 0.132ms 208547 256.70M 227.88M 39
10 master
10.1.2.208:63701 0.110ms 208567 256.65M 228.91M 40
13 master
10.1.2.208:63702 0.126ms 209087 256.39M 228.58M 37
14 master
10.1.2.208:63703 0.106ms 208364 256.75M 225.69M 38
14 master
10.1.2.208:63704 0.123ms 208780 257.42M 227.45M 41
15 master
10.1.2.208:63710 0.127ms 208448 256.74M 229.64M 38
14 master
10.1.2.208:63711 0.124ms 207825 256.21M 230.62M 40
13 master
10.1.2.208:63712 0.119ms 209181 256.74M 228.14M 32
13 master
10.1.2.208:63713 0.132ms 208890 255.82M 230.18M 34
13 master
10.1.2.208:63714 0.135ms 208991 157.47M 147.83M 23
9 master
10.1.2.209:63700 0.142ms 209062 219.46M 204.79M 40
17 slave
10.1.2.209:63701 0.126ms 207861 218.61M 190.75M 41
22 slave
10.1.2.209:63702 0.140ms 208486 218.08M 192.50M 40
12 slave
10.1.2.209:63703 0.127ms 208047 217.94M 191.14M 40
11 slave
10.1.2.209:63704 0.128ms 208136 218.55M 190.40M 42
11 slave
10.1.2.209:63710 0.127ms 208547 218.38M 196.05M 42
15 slave
10.1.2.209:63711 0.147ms 208567 218.83M 196.55M 43
13 slave
10.1.2.209:63712 0.133ms 209087 219.24M 197.19M 41
11 slave
10.1.2.209:63713 0.132ms 208364 218.35M 196.68M 42
16 slave
10.1.2.209:63714 0.135ms 208780 218.80M 196.15M 40
15 slave
10.1.2.210:63700 0.135ms 207757 254.64M 221.63M 39
16 master
10.1.2.210:63701 0.120ms 208971 257.44M 227.80M 40
16 master
10.1.2.210:63702 0.163ms 209144 257.02M 231.55M 41
14 master
10.1.2.210:63703 0.138ms 209009 257.39M 227.63M 39
15 master
10.1.2.210:63704 0.125ms 209095 257.35M 223.91M 41
15 master
10.1.2.210:63705 0.136ms 209037 256.63M 227.74M 39
14 master
10.1.2.210:63706 0.131ms 208478 255.19M 226.62M 39
17 master
10.1.2.210:63710 0.127ms 209062 257.87M 236.33M 41
19 master
10.1.2.210:63711 0.122ms 207861 255.21M 222.44M 39
21 master
10.1.2.210:63712 0.128ms 208486 255.86M 224.68M 40
12 master
10.1.2.210:63713 0.125ms 208047 256.46M 222.67M 40
14 master
10.1.2.210:63714 0.130ms 208136 255.90M 223.52M 39
12 master
10.1.2.211:63700 0.105ms 208787 219.07M 191.36M 40
15 slave
10.1.2.211:63701 0.104ms 208536 218.37M 194.23M 43
16 slave
10.1.2.211:63702 0.105ms 209121 219.19M 193.74M 41
12 slave
10.1.2.211:63703 0.104ms 208945 218.60M 194.22M 41
14 slave
10.1.2.211:63704 0.109ms 208343 218.05M 193.75M 41
12 slave
10.1.2.211:63705 0.129ms 208485 217.80M 193.87M 41
15 slave
10.1.2.211:63706 0.125ms 207975 217.90M 191.52M 41
13 slave
10.1.2.211:63710 0.122ms 207757 217.05M 190.80M 40
13 slave
10.1.2.211:63711 0.129ms 208971 218.65M 194.46M 43
13 slave
10.1.2.211:63712 0.104ms 209144 219.24M 200.27M 41
12 slave
10.1.2.211:63713 0.105ms 209009 219.22M 193.02M 40
14 slave
10.1.2.211:63714 0.104ms 209095 219.00M 191.43M 40
13 slave
10.1.2.211:63715 0.122ms 209037 218.94M 192.92M 40
12 slave
10.1.2.211:63716 0.104ms 208478 216.83M 193.62M 40
11 slave
10.1.2.212:63700 0.106ms 207943 255.54M 224.45M 39
15 master
10.1.2.212:63701 0.105ms 208386 256.36M 239.73M 40
14 master
10.1.2.212:63702 0.115ms 208219 255.92M 224.33M 41
18 master
10.1.2.212:63703 0.104ms 208123 256.87M 232.62M 43
16 master
10.1.2.212:63704 0.102ms 208095 255.89M 226.02M 40
17 master
10.1.2.212:63710 0.104ms 208787 256.54M 224.12M 40
15 master
10.1.2.212:63711 0.104ms 208536 256.46M 226.85M 40
16 master
10.1.2.212:63712 0.104ms 209121 257.63M 223.85M 40
13 master
10.1.2.212:63713 0.104ms 208945 256.44M 228.98M 41
14 master
10.1.2.212:63714 0.143ms 208343 255.59M 226.23M 39
13 master
10.1.2.212:63715 0.103ms 208485 256.17M 226.72M 40
14 master
10.1.2.212:63716 0.105ms 207975 255.95M 225.45M 43
11 master
AVERAGE: 0.139ms 208602 236.78M 210.70M 40
14 -
TOTAL: 17.778ms 26701078 29.60G 26.34G 5176
1853 master[64]/slave[64]
Original comment by daniel.m...@gmail.com
on 16 Jun 2011 at 3:20
Its is strange that redis is reporting used_memory higher than used_memory_rss.
Original comment by daniel.m...@gmail.com
on 16 Jun 2011 at 3:23
After a few minutes, the memory usage got to its previous state:
INSTANCE PING db0:keys mem mem_real clients
reqs/s role
10.1.2.201:63700 0.209ms 208769 58.11M 61.82M 37
14 master
10.1.2.201:63701 0.139ms 208959 58.19M 62.14M 37
11 master
10.1.2.201:63702 0.141ms 208831 58.01M 63.16M 37
11 master
10.1.2.201:63703 0.139ms 208767 58.11M 61.37M 37
9 master
10.1.2.201:63704 0.133ms 208590 57.98M 62.96M 37
11 master
10.1.2.201:63710 0.138ms 207943 57.93M 62.89M 38
15 slave
10.1.2.201:63711 0.139ms 208386 58.03M 61.74M 38
15 slave
10.1.2.201:63712 0.131ms 208219 58.05M 62.66M 38
12 slave
10.1.2.201:63713 0.131ms 208123 57.96M 62.68M 38
16 slave
10.1.2.201:63714 0.130ms 208095 57.94M 60.80M 38
11 slave
10.1.2.202:63700 0.133ms 208920 58.07M 62.00M 37
13 master
10.1.2.202:63701 0.128ms 207886 57.89M 61.55M 37
14 master
10.1.2.202:63702 0.131ms 208815 58.13M 62.62M 37
13 master
10.1.2.202:63703 0.128ms 208891 58.19M 62.29M 37
11 master
10.1.2.202:63704 0.128ms 209182 58.26M 62.61M 37
12 master
10.1.2.202:63710 0.126ms 208769 58.12M 60.73M 38
15 slave
10.1.2.202:63711 0.111ms 208959 58.20M 62.75M 38
11 slave
10.1.2.202:63712 0.125ms 208831 58.02M 60.84M 38
12 slave
10.1.2.202:63713 0.126ms 208767 58.13M 60.78M 39
9 slave
10.1.2.202:63714 0.129ms 208590 57.99M 62.33M 38
13 slave
10.1.2.203:63700 0.111ms 208014 57.92M 62.32M 38
14 slave
10.1.2.203:63701 0.110ms 209169 58.34M 62.34M 39
14 slave
10.1.2.203:63702 0.126ms 208939 58.18M 62.96M 38
12 slave
10.1.2.203:63703 0.111ms 208072 57.92M 62.41M 38
14 slave
10.1.2.203:63704 0.111ms 208414 58.10M 62.05M 38
13 slave
10.1.2.203:63710 0.111ms 208920 58.08M 61.17M 38
11 slave
10.1.2.203:63711 0.136ms 207886 57.90M 61.51M 38
15 slave
10.1.2.203:63712 0.110ms 208815 58.14M 61.82M 38
14 slave
10.1.2.203:63713 0.124ms 208891 58.20M 63.21M 38
11 slave
10.1.2.203:63714 0.109ms 209182 58.27M 62.02M 38
13 slave
10.1.2.204:63700 0.110ms 208450 58.02M 62.29M 38
12 slave
10.1.2.204:63701 0.110ms 208960 58.08M 62.05M 38
8 slave
10.1.2.204:63702 0.127ms 208211 58.03M 62.37M 38
15 slave
10.1.2.204:63703 0.129ms 209791 58.39M 62.75M 38
14 master
10.1.2.204:63704 0.110ms 208643 58.06M 63.03M 38
13 slave
10.1.2.204:63710 0.109ms 208014 57.91M 61.75M 37
13 master
10.1.2.204:63711 0.109ms 209169 58.33M 62.74M 37
12 master
10.1.2.204:63712 0.110ms 208939 58.18M 63.13M 38
16 master
10.1.2.204:63713 0.126ms 208072 57.92M 62.90M 38
16 master
10.1.2.204:63714 0.131ms 208414 58.10M 62.02M 37
13 master
10.1.2.205:63700 0.148ms 208903 58.15M 63.19M 38
13 slave
10.1.2.205:63701 0.133ms 209233 58.19M 61.92M 38
14 slave
10.1.2.205:63702 0.142ms 208794 58.09M 62.81M 38
11 slave
10.1.2.205:63703 0.129ms 208086 57.88M 62.22M 38
10 slave
10.1.2.205:63704 0.134ms 208391 58.04M 61.08M 38
9 slave
10.1.2.205:63710 0.138ms 208450 58.02M 60.72M 38
12 master
10.1.2.205:63711 0.137ms 208960 58.07M 61.70M 37
12 master
10.1.2.205:63712 0.135ms 208211 58.02M 62.29M 37
19 master
10.1.2.205:63713 0.136ms 209791 58.38M 61.40M 38
14 slave
10.1.2.205:63714 0.147ms 208643 58.05M 63.17M 37
12 master
10.1.2.206:63700 0.161ms 207725 57.82M 61.89M 37
12 master
10.1.2.206:63701 0.176ms 208958 58.23M 63.14M 37
11 master
10.1.2.206:63702 0.132ms 208266 57.90M 62.87M 38
15 master
10.1.2.206:63703 0.133ms 208515 58.04M 62.02M 37
12 master
10.1.2.206:63704 0.151ms 208674 58.09M 62.57M 38
12 master
10.1.2.206:63710 0.140ms 208903 58.14M 62.50M 37
16 master
10.1.2.206:63711 0.136ms 209233 58.20M 63.32M 39
14 master
10.1.2.206:63712 0.137ms 208794 58.09M 62.73M 37
14 master
10.1.2.206:63713 0.131ms 208086 57.88M 61.67M 37
9 master
10.1.2.206:63714 0.130ms 208391 58.03M 61.66M 37
12 master
10.1.2.207:63700 0.085ms 208448 58.07M 60.77M 38
13 slave
10.1.2.207:63701 0.057ms 207825 57.92M 62.25M 38
11 slave
10.1.2.207:63702 0.056ms 209181 58.26M 62.23M 38
8 slave
10.1.2.207:63703 0.061ms 208890 58.06M 61.54M 38
10 slave
10.1.2.207:63704 0.053ms 208991 58.09M 62.55M 39
11 slave
10.1.2.207:63710 0.057ms 207725 57.83M 60.74M 38
11 slave
10.1.2.207:63711 0.056ms 208958 58.24M 61.84M 38
12 slave
10.1.2.207:63712 0.061ms 208266 57.90M 61.41M 38
12 slave
10.1.2.207:63713 0.060ms 208515 58.04M 62.44M 38
11 slave
10.1.2.207:63714 0.070ms 208674 58.09M 62.10M 38
11 slave
10.1.2.208:63700 0.177ms 208547 57.99M 61.75M 37
11 master
10.1.2.208:63701 0.124ms 208567 58.13M 62.24M 37
13 master
10.1.2.208:63702 0.121ms 209087 58.26M 62.99M 38
8 master
10.1.2.208:63703 0.129ms 208364 58.01M 61.77M 37
11 master
10.1.2.208:63704 0.125ms 208780 58.06M 62.57M 37
13 master
10.1.2.208:63710 0.134ms 208448 58.07M 62.11M 37
14 master
10.1.2.208:63711 0.124ms 207825 57.91M 61.87M 37
14 master
10.1.2.208:63712 0.129ms 209181 58.24M 62.91M 37
14 master
10.1.2.208:63713 0.123ms 208890 58.05M 61.91M 37
9 master
10.1.2.208:63714 0.121ms 208991 58.08M 61.81M 37
12 master
10.1.2.209:63700 0.117ms 209062 58.16M 61.90M 39
9 slave
10.1.2.209:63701 0.104ms 207861 57.91M 60.86M 39
13 slave
10.1.2.209:63702 0.124ms 208486 58.05M 62.41M 38
12 slave
10.1.2.209:63703 0.122ms 208047 58.02M 61.35M 38
14 slave
10.1.2.209:63704 0.108ms 208136 57.94M 61.07M 38
13 slave
10.1.2.209:63710 0.105ms 208547 58.01M 61.57M 39
11 slave
10.1.2.209:63711 0.123ms 208567 58.14M 62.58M 38
11 slave
10.1.2.209:63712 0.122ms 209087 58.26M 63.58M 38
9 slave
10.1.2.209:63713 0.122ms 208364 58.03M 62.40M 39
12 slave
10.1.2.209:63714 0.124ms 208780 58.06M 62.62M 38
15 slave
10.1.2.210:63700 0.133ms 207757 57.80M 60.68M 37
13 master
10.1.2.210:63701 0.130ms 208971 58.17M 61.72M 37
12 master
10.1.2.210:63702 0.108ms 209144 58.22M 62.12M 38
13 master
10.1.2.210:63703 0.127ms 209009 58.14M 62.03M 37
13 master
10.1.2.210:63704 0.129ms 209095 58.13M 61.57M 37
13 master
10.1.2.210:63705 0.113ms 209037 58.15M 61.08M 37
10 master
10.1.2.210:63706 0.132ms 208478 57.98M 62.96M 37
14 master
10.1.2.210:63710 0.130ms 209062 58.15M 60.73M 37
10 master
10.1.2.210:63711 0.132ms 207861 57.89M 61.91M 37
11 master
10.1.2.210:63712 0.130ms 208486 58.04M 62.53M 37
14 master
10.1.2.210:63713 0.130ms 208047 58.01M 63.06M 37
11 master
10.1.2.210:63714 0.136ms 208136 57.93M 60.96M 38
11 master
10.1.2.211:63700 0.109ms 208787 58.26M 62.81M 38
13 slave
10.1.2.211:63701 0.107ms 208536 58.04M 62.09M 38
13 slave
10.1.2.211:63702 0.107ms 209121 58.21M 61.96M 38
14 slave
10.1.2.211:63703 0.106ms 208945 58.09M 61.58M 38
10 slave
10.1.2.211:63704 0.107ms 208343 57.96M 62.05M 38
11 slave
10.1.2.211:63705 0.105ms 208485 58.06M 62.93M 38
10 slave
10.1.2.211:63706 0.108ms 207975 57.96M 62.92M 38
15 slave
10.1.2.211:63710 0.108ms 207757 57.81M 60.37M 38
12 slave
10.1.2.211:63711 0.107ms 208971 58.18M 61.21M 38
13 slave
10.1.2.211:63712 0.108ms 209144 58.22M 61.34M 38
13 slave
10.1.2.211:63713 0.107ms 209009 58.15M 62.71M 38
16 slave
10.1.2.211:63714 0.106ms 209095 58.13M 62.75M 38
12 slave
10.1.2.211:63715 0.107ms 209037 58.16M 61.23M 38
11 slave
10.1.2.211:63716 0.113ms 208478 57.99M 62.25M 38
12 slave
10.1.2.212:63700 0.106ms 207943 57.93M 62.39M 38
9 master
10.1.2.212:63701 0.106ms 208386 58.02M 62.79M 37
14 master
10.1.2.212:63702 0.112ms 208219 58.03M 61.49M 37
13 master
10.1.2.212:63703 0.106ms 208123 57.96M 63.39M 37
14 master
10.1.2.212:63704 0.107ms 208095 57.93M 61.50M 37
15 master
10.1.2.212:63710 0.106ms 208787 58.26M 62.25M 37
12 master
10.1.2.212:63711 0.105ms 208536 58.03M 62.08M 37
11 master
10.1.2.212:63712 0.105ms 209121 58.22M 61.39M 38
13 master
10.1.2.212:63713 0.107ms 208945 58.08M 63.06M 37
12 master
10.1.2.212:63714 0.106ms 208343 57.96M 62.23M 37
9 master
10.1.2.212:63715 0.106ms 208485 58.06M 62.51M 38
10 master
10.1.2.212:63716 0.134ms 207975 57.95M 62.66M 37
11 master
AVERAGE: 0.119ms 208602 58.07M 62.11M 37
12 -
TOTAL: 15.266ms 26701078 7.26G 7.76G 4821
1575 master[64]/slave[64]
Original comment by daniel.m...@gmail.com
on 16 Jun 2011 at 3:28
The "top" command showing some of the instances after the memory returned to
its previous state:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ SWAP P TIME COMMAND
12949 root 5 -10 170m 63m 664 S 0.0 0.4 0:24.78 107m 2 0:24
redis_B0_47 /etc/redis/redis_B0_47.conf
12944 root 5 -10 158m 63m 672 S 0.0 0.4 0:24.84 95m 2 0:24
redis_B0_46 /etc/redis/redis_B0_46.conf
12955 root 5 -10 174m 63m 680 S 0.0 0.4 0:24.78 111m 2 0:24
redis_B1_41 /etc/redis/redis_B1_41.conf
12948 root 5 -10 162m 63m 672 S 0.0 0.4 0:24.70 99m 2 0:24
redis_B0_48 /etc/redis/redis_B0_48.conf
12947 root 5 -10 198m 62m 664 S 0.0 0.4 0:24.65 135m 2 0:24
redis_B0_51 /etc/redis/redis_B0_51.conf
12946 root 5 -10 170m 62m 668 S 0.0 0.4 0:24.79 107m 2 0:24
redis_B0_49 /etc/redis/redis_B0_49.conf
12952 root 5 -10 194m 62m 668 S 0.0 0.4 0:24.46 132m 2 0:24
redis_B1_44 /etc/redis/redis_B1_44.conf
12951 root 5 -10 146m 62m 660 S 0.0 0.4 0:24.64 84m 2 0:24
redis_B0_50 /etc/redis/redis_B0_50.conf
12950 root 5 -10 166m 61m 660 S 0.0 0.4 0:24.37 104m 2 0:24
redis_B1_42 /etc/redis/redis_B1_42.conf
12945 root 5 -10 170m 61m 668 S 0.0 0.4 0:24.41 108m 2 0:24
redis_B0_45 /etc/redis/redis_B0_45.conf
12954 root 5 -10 142m 61m 664 S 0.0 0.4 0:24.67 80m 2 0:24
redis_B1_40 /etc/redis/redis_B1_40.conf
12953 root 5 -10 142m 61m 660 S 0.0 0.4 0:24.44 81m 2 0:24
redis_B1_43 /etc/redis/redis_B1_43.conf
Original comment by daniel.m...@gmail.com
on 16 Jun 2011 at 3:36
That is indeed weird.. Not sure why that is happening.. What I forgot to
mention in my previous post is that jemalloc only works with 64-bit binaries. I
see in your top-post that you use a 32-bit build. The 32-bit build always uses
a little less memory than the 64-bit build because of the smaller pointer
overhead.
Building a 64-bit version of Redis with jemalloc is not done by default, but
requires setting an env var:
USE_JEMALLOC=yes make
By the way: I would try this scenario on a development machine, or as single
instance side-by-side with your cluster before deploying this everywhere. You
never know what happens for a particular OS/dataset/config.
Original comment by pcnoordh...@gmail.com
on 16 Jun 2011 at 3:41
I have built Redis like this:
make 32bit USE_JEMALLOC=yes
And changed the Makefile to this:
../deps/jemalloc/lib/libjemalloc.a:
cd ../deps/jemalloc && ./configure CFLAGS="-std=gnu99 -Wall -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -m32" --with-jemalloc-prefix=je_ --enable-cc-silence && $(MAKE) lib/libjemalloc.a
I'm comfortable using the jemalloc branch in 32 bits mode since I have already
use it for quite some time a few weeks ago and it seemed to work fine.
Here is my INFO using jemalloc int 32 bits mode:
redis_version:2.2.8
redis_git_sha1:00000000
redis_git_dirty:0
arch_bits:32
multiplexing_api:epoll
process_id:5799
uptime_in_seconds:3544
uptime_in_days:0
lru_clock:800489
used_cpu_sys:32.79
used_cpu_user:6.63
used_cpu_sys_childrens:3.71
used_cpu_user_childrens:1.06
connected_clients:37
connected_slaves:1
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
used_memory:60960148
used_memory_human:58.14M
used_memory_rss:65716224
mem_fragmentation_ratio:1.08
mem_allocator:jemalloc-2.2.1
loading:0
aof_enabled:0
changes_since_last_save:0
bgsave_in_progress:0
last_save_time:1308238911
bgrewriteaof_in_progress:0
total_connections_received:422
total_commands_processed:903752
expired_keys:0
evicted_keys:0
keyspace_hits:483398
keyspace_misses:183
hash_max_zipmap_entries:8
hash_max_zipmap_value:2048
pubsub_channels:0
pubsub_patterns:0
vm_enabled:0
role:master
allocation_stats:...
db0:keys=208786,expires=0
Original comment by daniel.m...@gmail.com
on 16 Jun 2011 at 3:51
Cool! I noticed a couple compiler warnings about a field not being large
enough, so I assumed it would crash hard when compiled with -m32. So: changing
to jemalloc resulted in the memory being reclaimed by the OS after some time,
correct?
Original comment by pcnoordh...@gmail.com
on 16 Jun 2011 at 4:13
Redis 2.4 can also be compiled with 32 bit target plus jemalloc, just with:
make 32bit.
I tried a few days ago and it passess all the tests without issues.
Salvatore
Original comment by anti...@gmail.com
on 16 Jun 2011 at 4:28
Btw I think the issue is just the result of a memory spike that is not
reclaimed by the allocator, but will be used again when the spike will happen
again, so should not be a big deal at all. Even if it is cooler an allocator
that is also able to effectively reclaim memory... of course :)
Original comment by anti...@gmail.com
on 16 Jun 2011 at 4:29
Well, I think I'll have to use the MONITOR command with parsimony then :)
Thanks for the discussion.
Original comment by daniel.m...@gmail.com
on 17 Jun 2011 at 6:42
Daniel: you don't need that actually! What you need is to split your processing
in two halves. One reads from MONITOR without waiting for processing, and
writing all the MONITOR output into a file. The other one reads the file
processing the output. This way your processing can be slower than the MONITOR
output without issues.
Original comment by anti...@gmail.com
on 18 Jun 2011 at 8:57
Swapping in jemalloc appeared to do the trick here. Closing.
Original comment by pcnoordh...@gmail.com
on 6 Jul 2011 at 12:08
Original issue reported on code.google.com by
daniel.m...@gmail.com
on 15 Jun 2011 at 9:11