What versions does webdis require

[digitalrinaldo]

I can no longer get webdis to work redis.

Latest trunk of webdis I upgraded redis to 2.6 I upgraded libevent to 2.0.so.5.1.4

I am getting 503 errors under the slightest load.

[nicolasff]

Hi,

This is unexpected; I will have a go at reproducing this issue. Do you see any errors in the webdis logs? If not, could you increase the log level?

Thank.

[digitalrinaldo]

Thanks -- btw this a great piece of work nothing out there comes close. I increased the log level and tried 10,20,50,100 threads and the same issue happens in the log "Too many connections" I have tried everything that I can think of 1) Setting keep alive 2) changed the c code to use keep alive 3) smaller loads with a delay ( using the request library) 4) went back to 2.2 and 1.4 libevent same issue

I am trying to load 2,000 records using request and it seems to be opening new connections for each I see there is quite a bit of confusion in the node community about http.agent and pooling been looking at this for 3 days and trying to decide what to do

[digitalrinaldo]

Attempting 2000+ g GET request using var request = require('request'); The log is full of the following.

[8749] 22 Apr 13:37:12 0 /3/SET/........... [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 503 [8749] 22 Apr 13:37:12 0 /3/SET/......

The client code is, It seems like a very small load to me (2000+ requests in a few seconds)

reader.addListener('data', function(data) { var key = data.Code;

var description = JSON.stringify(data); var options = { url: "http://server:7379/3/SET/key-name:" + key + "/" + encodeURIComponent(description), sendImmediately: true, auth: { user: "user", pass: "password" } }; //options.agent = new https.Agent(options); request(options, function(error, response, body) { recordsWritten++; if ((error != null) || (response.statusCode != 200)) { console.log(error, body); logger.fatal("Failed to write:" + url); } else { //logger.info("Wrote:" + util.inspect(response.request.uri.href)); } callBack(); }); });

Redis config

{ "redis_host": "127.0.0.1", "redis_port": 6379, "redis_auth": "password", "http_host": "0.0.0.0", "http_port": 7379, "threads": 50, "database": 0,

ldd on executable /opt/webdis/webdis

linux-vdso.so.1 =>  (0x00007fffdbdfe000)
libevent-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5 (0x00007f1278d0f000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1278af2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1278729000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1278521000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1278f5a000)

redis-server -v Redis server v=2.6.7 sha=00000000:0 malloc=jemalloc-3.0.0 bits=64

I don't think webdis has a version number -- latest trunk with git clone then make ; make install

[nicolasff]

Thanks a lot, I'll have a go at reproducing it tonight.

[digitalrinaldo]

Tried this on LTS 12.04

and 13

Linux ip-10-140-75-26 3.8.0-16-generic #26-Ubuntu SMP Mon Apr 1 19:52:57 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

[nicolasff]

These error messages are probably due to webdis creating too many connections to Redis; this happens because you are sending commands to a Redis "database" (three) which is not the default one (zero). Webdis keeps a pool of connections to Redis, but they are all connected to the same default DB. When a non-default DB is required, webdis opens a new connection to Redis. Webdis should be closing connections, I will check that this is indeed the case and add more logs regarding failed connections to Redis.

[nicolasff]

@digitalrinaldo Thanks for opening this ticket, there was indeed an issue with Webdis not closing open connections on non-default DBs. Could you please try out the latest master?

Creating a large number of connections to Redis is still not ideal and you might run out of file descriptors quickly. Try configuring webdis to use only one DB and stick to it if possible.

Thanks.