Redis server unreachable when uploading a dataset of 800,000 key/value pairs.

[GoogleCodeExporter]

What version of Redis you are using, in what kind of Operating System?
v. 2.2.11, Ubuntu Lucid Lynx (10.04) 

What is the problem you are experiencing?
I am trying to upload an example dataset with about 800,000 key value pairs 
into a sorted Redis set using zadd.   My python script (redis-py v 2.2.8), 
invariably fails when uploading the 28233 item after which the redis-server 
becomes unavailable (I cannot telnet to it).  The server is back again after a 
while.

What steps will reproduce the problem?
Upload key value pairs using a script. The script is unable to connect to the 
server after a while fails with a connection error. 

Do you have an INFO output? Please past it here.

If it is a crash, can you please paste the stack trace that you can find in
the log file or on standard output? This is really useful for us!

Please provide any additional information below.

Original issue reported on code.google.com by sriharsh...@gmail.com on 22 Jul 2011 at 8:03

[GoogleCodeExporter]

Are the commands sent in a pipeline, or part of a MULTI/EXEC block? It would 
help if you provided an excerpt of the commands that are sent to Redis, or even 
the script you use when you observe this behavior. When the issue is 
reproducible, it should not be very hard to track down what is causing this. 
Thanks.

Original comment by pcnoordh...@gmail.com on 22 Jul 2011 at 8:08

[GoogleCodeExporter]

No, the commands are not sent in a pipeline nor is it a part of a MULTI/EXEC 
block. 

The script is very simple: Key/Value pairs are read from a plain text file and 
zadd is continuously called until EOF. 

Original comment by sriharsh...@gmail.com on 22 Jul 2011 at 8:38

[GoogleCodeExporter]

Attaching the script.

Original comment by sriharsh...@gmail.com on 22 Jul 2011 at 8:42

Attachments:

• test.py

[GoogleCodeExporter]

You're creating a new connection for every ZADD. Probably Python can't GC the 
connections fast enough, and you run out of file descriptors. Use a single 
connection to perform all ZADD's instead.

Original comment by pcnoordh...@gmail.com on 22 Jul 2011 at 8:58

• Changed state: Invalid

[GoogleCodeExporter]

Using a single connection fixed it. I could add all the key/value pairs without 
a hitch. 

Original comment by sriharsh...@gmail.com on 22 Jul 2011 at 10:02