Very slow performance with a local server

[joelkuiper]

Hey,

I'm having a performance issue problem with running Redis (2.6) on Ubuntu 12.04.

After filling a database by executing the following command in a redis-cli

eval "for i=1,100 do redis.call('sadd', i, 1); end; return 1" 0

Which fills a 100 sets with one element. Then in R running the following commands:

library(rredis) redisConnect()

system.time (lapply(1:100, function(i) { redisSMembers(as.character(i)) })) user system elapsed 0.104 0.044 4.000

system.time (lapply(1:100, function(i) { runif(1) })) user system elapsed 0.000 0.000 0.001

The operation takes about 4 second elapsed time, which seems a bit long especially with regard to the user time ;-). Happens consistently as I just iterated over 400k sets with two Redis operations which lasted about 7,5 hours (26060.933 seconds to be precise).

redis-benchmark results in acceptable performance, so I'm kinda at a loss what's going here! It works fine on my Mac, happens with both 32 and 64 bit R versions (latest major release).

[bwlewis]

Dear Joel,

Sorry for the latency in responding...been pretty busy.

Yes, by default, the rredis package waits for a response from the redis server after each transaction. There are two approaches to dramatically improve performance in such contexts:

1. Use redis transactions
2. Use redisSetBlocking/redisGetResponse

Here is an example of the latter for your example:

library(rredis) redisConnect()

t1 = proc.time() x = lapply(1:100, function(i) { redisSMembers(as.character(i)) }) print(proc.time() - t1)

redisSetBlocking(FALSE)

t1 = proc.time() lapply(1:100, function(i) { redisSMembers(as.character(i)) }) y = redisGetResponse() print(proc.time() - t1)

Does that help?

[bwlewis]

If this didn't help for you, just let me know and re-open this issue.

--Bryan