arjunmehta / node-georedis

Super fast geo queries.
MIT License
192 stars 33 forks source link

Get nearby locations timeouts the server with 10km radius #30

Closed MadeinFrance closed 4 years ago

MadeinFrance commented 4 years ago

I retrieve locations with the nearby method. However I see the following errors:

{ AbortError: GEORADIUS can't be processed. The connection is already closed.}
at handle_offline_command (/srv/node_modules/redis/index.js:851:15) 
at RedisClient.internal_send_command (/srv/node_modules/redis/index.js:885:9) 
at RedisClient.georadius (/srv/node_modules/redis/lib/commands.js:58:25) 
at RedisClient.send_command.RedisClient.sendCommand (/srv/node_modules/redis/lib/extendedApi.js:46:26) 
at georadiusGeneric (/srv/node_modules/georedis/lib/interfaceNative.js:178:10) 
at NativeInterface.georadius (/srv/node_modules/georedis/lib/interfaceNative.js:124:3) 
at GeoSet.nearby (/srv/node_modules/georedis/main.js:146:31) ) 
at new Promise (<anonymous>) 
at process._tickCallback (internal/process/next_tick.js:68:7) 
command: 'GEORADIUS', 
code: 'NR_CLOSED', 

args: 
 [ 'geo', 
   -87.75563344016012, 
   41.89034192093163, 
   10000, 
  'm' ] } 

There's 250K locations saved.

Other simple redis commands works. Should I consider using another way of storing locations? Any ideas of the cause?

By reducing the radius to 5km it seems to happen less.

arjunmehta commented 4 years ago

@MadeinFrance Thanks for reporting this potential bug.

Could you tell me what version of the redis module you are using?

It seems like this may be an issue with the redis module when it fails to retry. See here: https://github.com/NodeRedis/node_redis/issues/1418

My very naive guess is that redis is returning a large number of results and that is taking some time to parse, and so it is dropping the connection/retrying/failing to retry.

It seems like the issue may not present itself when using ioredis. Would you like to give that a try?

MadeinFrance commented 4 years ago

Redis server v=5.0.3 sha=00000000:0 malloc=libc bits=64 build=5ac74f5d434534dd

I can give a try with ioredis.

So node-georedis is compatible with ioredis?

var Redis = require("ioredis");
var redisClient = new Redis();
const geo = require('georedis').initialize(redisClient);
arjunmehta commented 4 years ago

@MadeinFrance Yes that would be exactly how to use ioredis, that should work the same as with the standard redis package.

As for the version of redis, I meant which version of the redis npm package.

MadeinFrance commented 4 years ago

"redis": "2.8.0" "georedis": "3.1.1",

I will update this issue with my results of ioredis

arjunmehta commented 4 years ago

@MadeinFrance Okay thank you. Please keep me posted.

Can you tell me, when performing your original query at 10km, is there a delay before the error occurs? How many results of the 250,000 are expected at 10km? How many at 5km?

You mention that the issue happens less at a smaller radius, but it still happens?

MadeinFrance commented 4 years ago

Loading 10km returns ~60 results which is handled well.

Thanks for pointing at it, using ioredis fixed the timeouts issues.

Closing the issue since is not related to this well made library.

arjunmehta commented 4 years ago

@MadeinFrance Okay thank you for letting me know. If you want to continue using the redis module, you may want to look into the retry_strategy option, or also the socket_keepalive option. I'm not sure at all if it will solve your problem, but it seems like a probably place to look:

 const client = redis.createClient({
    socket_keepalive: true,
    retry_strategy: (options) => {
      if (options.error && options.error.code === 'ECONNREFUSED') {
        return new Error('The server refused the connection');
      }
      if (options.total_retry_time > 1000 * 60 * 60) {
        return new Error('Retry time exhausted');
      }
      if (options.attempt > 10) {
        return undefined;
      }

      // reconnect after
      return 1000;
    }
  });