Redis eating all cpu (system)

[GoogleCodeExporter]

Centos 5.3 x86_64, Redis 0.096

Run redis-server for some time under some load and see

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
22602 memd      25   0  4192  916  388 R 85.4  0.0  29:54.39 redis-server

At this time redis did not accept new connections, but write in log-file 
. DB 0: 139 keys (0 volatile) in 256 slots HT.
. 50 clients connected (0 slaves), 97086 bytes in use
(numbers did not changing)

I restarted redis-server and repeated this situation few times with strace
running. Results in attachment (1.3 Mb).

Original issue reported on code.google.com by vereteln...@gmail.com on 22 May 2009 at 2:19

Attachments:

• redis_long.txt

[GoogleCodeExporter]

Hello, thank you very much for the bug report, could you please provide the 
exact
Redis configuration and the script you used to stress test it? Thanks

Original comment by anti...@gmail.com on 22 May 2009 at 2:30

[GoogleCodeExporter]

Never mind, was able to reproduce the issue myself, this was not discovered 
before
because is related to command pipelining that is rarely used. Ok let's fix 
this...

Original comment by anti...@gmail.com on 22 May 2009 at 2:46

• Added labels: Priority-Critical
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Antirez, do you need stress script yet? I have no such at this moment. We are 
using 
redis in semi-production enviroment. May be problem appears due to some network 
issues between client and server?

Config:

daemonize yes
pidfile /home/memd/redis/var/run/redis.pid
port 6379
timeout 300
save 900 1
save 300 10
save 60 10000
dir /home/memd/redis/var/lib/redis/
loglevel debug
logfile /home/memd/redis/var/log/redis.log
databases 1
glueoutputbuf yes
shareobjects no

Original comment by vereteln...@gmail.com on 22 May 2009 at 3:00

[GoogleCodeExporter]

No it's just a problem with command pipelining, I can fix it with the test 
script I
wrote myself. This was a valuable bugfix, thank you, I'm already debugging the 
issue
to fix it. The fix will be trivial, what is not is to find the exact conditions
generating the issue :) A matter of some hour and I'll kill this nasty bug! 
.... ;)

Original comment by anti...@gmail.com on 22 May 2009 at 3:04

[GoogleCodeExporter]

Ok problem tracked. Quick fix: "glueoutputbuffers no" in redis.conf.

Proper fix in the next hours.

Cheers,
Salvatore

Original comment by anti...@gmail.com on 22 May 2009 at 4:10

[GoogleCodeExporter]

Bug fixed in Redis-git, thanks for the help.

Original comment by anti...@gmail.com on 22 May 2009 at 4:36

• Changed state: Verified

[GoogleCodeExporter]

Is it correct to apply patch to Redis 0.096? Or it's better to wait for next 
release?

Original comment by vereteln...@gmail.com on 22 May 2009 at 5:32

[GoogleCodeExporter]

For 0.096 the best thing to do is to set glueoutputbuffer to no in the 
redis.conf
file, btw a new tar.gz, redis 0.100 will be released in one/two days at max. 
Cheers

Original comment by anti...@gmail.com on 22 May 2009 at 5:37