search

tidwall / tile38

Real-time Geospatial and Geofencing
https://tile38.com
MIT License
9.15k stars 570 forks source link

NearBy Hook is relatively slow #488

Open abhit011 opened 5 years ago

abhit011 commented 5 years ago

I have done NearBy hooks for close to 50K static points with condition where masterID
and One of the typical hooks look like this 'NEARBY','fleet','DISTANCE','WHERE','masterID',201,201,'FENCE','DETECT','inside','POINT',lat,long,'100000'

But when I am doing setpoint of close to 30k IMEIS ,the setpoint loop itself takes close to 20 minutes to just complete sending to tile38
redisClient.Do("set", "fleet",imei,"field","sq", s,"field", "masterID", anotherID,"point", laT, loN)

Tried sending parallel ,but its same ..

Idea is to find the minimum nearby distance of each IMEI against respective masterID hook. if an iteration of 30K set points is going to take 20 minutes, am i doing something wrong ?

abhit011 commented 5 years ago

Whether I do the Setpoint first and then call the Nearby over set of static point with 100KM radius , results are really low.. It takes close to lot of times to find the neayby for just 30k static points over 50K IDs

Does it ?

abhit011 commented 5 years ago

Any suggestions @tidwall

abhit011 commented 5 years ago

The results are very disheartening , am i doing something wrong ?

tidwall commented 5 years ago

Hi @abhit001, I have a couple of ideas off the top of my head. First is exploring an alternative geofence type. The NEARBY command has extra overhead compared to INTERSECTS or WITHIN. Perhaps try INTERSECTS ... FENCE ... CIRCLE lat long 100000. Another thing to look into is command pipelining. Depending on your workflow, this may or may not be available to you, but Tile38 is optimized for accepting multiple SET commands in a single packet. My understanding is that the Go Redis clients support pipelining.

abhit011 commented 5 years ago

@tidwall , tried using INTERSECTS with Circle along with redis pipline using'redis.Send', but trust me the results were more or less the same ...