Very slow performance for data size greater than 8k while connecting to localhost

[GoogleCodeExporter]

What steps will reproduce the problem?

1.ensure redis-server is running on your own computer
2.extract the attached archive (php source code)
3.modify the test.php file, change the server ip address to your ip, or just 
127.0.0.1
4.run the test program by doing 'php test.php'.  Supplying the script with a 
file of over 8kb as the data to store into redis.

What is the expected output? What do you see instead?

The performance should be well over thousands operations for this simple test.  
But instead, you simply get stuck with large data. However, the same data size 
is fine if redis is running on remote host!  

Note: I found that if data is under 8kb this problem does not occure, but if 
over 8kb, this problem has a very high probability (e.g. over 90%) of occuring. 
Still there are some case that it does not appear, for example I do a test with 
a file of about 25kb, it was fine, but if I use split to chop a 16kb data from 
the same file, the bug occured.

What version of the product are you using? On what operating system?

redis version 2.0.1 running on Ubuntu 10.04 amd64.

Original issue reported on code.google.com by xrf...@gmail.com on 14 Sep 2010 at 11:22

Attachments:

• php-redis.tar.gz

[GoogleCodeExporter]

It could be that you have a slow PHP client.

I tried out redis-node-client, it's over 800 lines and took 440ms to get a 
1.5mb value out of Redis.

I wrote my own Redis client in 150 lines and it takes 15ms to get the same 
1.5mb value out of Redis.

If you suspect your client is slow, check the code where it buffers the value. 
Does it wait to get the value in one go, or does it append a temporary value 
byte by byte? If it's the latter, it's probably slow and this would only be 
noticeable when testing with larger values.

Original comment by jorangr...@gmail.com on 14 Sep 2010 at 3:05

[GoogleCodeExporter]

The slowdown is due to a known limitation of PHP when using socket streams 
since the engine doesn't expose a way to set the TCP_NODELAY flag on the 
underlying socket resource. Every pure-PHP client is affected, not just Predis.

See http://bugs.php.net/bug.php?id=51879 for further details and 
http://github.com/nrk/predis/issues#issue/10 for a proposed solution with 
Predis (still under development).

Original comment by suppaki...@gmail.com on 14 Sep 2010 at 4:09

[GoogleCodeExporter]

Thanks for the info.  However, I observe this behavior:

1) the extreme slow down only happen when you connect to localhost (i.e. redis 
running on localhost)!

2) if data size lower than 4k, there is also a slowdown compare to localhost, 
e.g. while running on localhost with datasize=4k, the performance is about 8k 
rw/s, but running on another machine on the LAN, performance is about 2.5k 
rw/s, is that normal, or it is also a result of the php bug you mentioned?

Original comment by xrf...@gmail.com on 14 Sep 2010 at 10:27

[GoogleCodeExporter]

The same problem occur if I am using Jedis (the java driver).  Please see 
attached test program (a copy of the Jedis source code is also attached for the 
sake of review convenience).

I will also post this problem to Jedis mailing list, see if it is again a 
client problem?

Original comment by xrf...@gmail.com on 15 Sep 2010 at 8:15

Attachments:

• tredis.tar.gz
• Jedis.tar.gz

[GoogleCodeExporter]

1) the TCP stack is usually tuned differently for connections to the loopback 
interface, which means that you might get different behaviors out of the same 
tests between a connection to the localhost and an actual connection to a 
remote host.

2) it could be a mix of point 1 and the natural performance hit that is caused 
by the RTT (network round-trip time), which is obviously greater with 
connections to a remote endpoint. See 
http://groups.google.com/group/redis-db/msg/2669d6c13361db72 for a concise 
explanation of the cost of the RTT and how it can be avoided in certain cases 
by using pipelining (which is supported by most of the client libraries out 
there).

Original comment by suppaki...@gmail.com on 15 Sep 2010 at 8:26

[GoogleCodeExporter]

As for the performance "degradation" when testing over LAN: 2.5kops with 4kb 
payload equals 10MB/sec, which is close to the bandwidth of 100Mbit ethernet. 
So, I think you're simply reaching the bandwidth limit for point 2).

Original comment by pcnoordh...@gmail.com on 15 Sep 2010 at 8:33

[GoogleCodeExporter]

Can you please check if the same conditions hold (slow over loopback) if you 
use the tool redis-benchmark? Use the option "-d" to specify the size of the 
payload to use in the benchmark (and "--help" for all options).

If the output of these tests show the same behavior please report back and I 
the issue will be reopened. Otherwise, this is a client library issue and you 
should contact the authors of the library you are using. The Redis issues list 
is meant to be used for redis-server issues only. For now, I'm closing the 
issue because my guess is that this is a limitation of PHP or the client 
library.

Original comment by pcnoordh...@gmail.com on 15 Sep 2010 at 8:40

• Changed state: Invalid

[GoogleCodeExporter]

Hi pcnnordhuis,

1) I am using a Gigabit LAN, so I don't think there will be a bandwidth 
bottleneck. I will study the post pointed by suppakilla for this issue.

2) the issue does NOT occur if using redis-benchmark, so I think it is indeed a 
client lib info, I will trace predis and jedis accordingly.

BTW, I wonder if there is a mailing list for redis or not, and it seems that 
the irc channel is not very active so I assume that developers and users of 
redis has a better channel for communication?

As this ticket seems no needs to be reopened, if you please reply my above 
question directly to xrfang@gmail.com  Thank you very much!

Original comment by xrf...@gmail.com on 15 Sep 2010 at 9:54

[GoogleCodeExporter]

The 100Mbit figure was more of a guess (considering the number you got), but if 
you are not hitting Redis with multiple parallel clients, the RTT will probably 
be the cause of this (exactly what suppakilla says).

The mailing list can be found here: http://groups.google.com/group/redis-db/ . 
In my experience the IRC channel is *very* active (differs per timezone you are 
in I guess). Please feel free to ask away there.

Original comment by pcnoordh...@gmail.com on 15 Sep 2010 at 10:15

[GoogleCodeExporter]

Hello,

I write some code implement Rediska_Connection_Socket which use socket_* 
functions.

This code resolve the 8k bad performance problem.

and patch  Rediska.php
Index: Rediska.php
===================================================================
--- Rediska.php (revision 19336)
+++ Rediska.php (revision 19340)
@@ -236,7 +236,10 @@
         $options['host'] = $host;
         $options['port'] = $port;
-        $this->_connections[$connectionString] = new 
Rediska_Connection($options);
+       if(function_exists('socket_create'))
+         $this->_connections[$connectionString] = new 
Rediska_Connection_Socket($options);
+       else
+         $this->_connections[$connectionString] = new 
Rediska_Connection($options);
         if ($this->_keyDistributor) {
             $this->_keyDistributor->addConnection(

Original comment by ruanchun...@gmail.com on 22 Aug 2012 at 3:14

Attachments:

• Socket.php