Missing Results when records with Large ID length

[owaaa]

I'm seeing some odd issues with long(ish) id lengths. I have some multipart IDs I'm placing inside the same key in order to do a NEARBY search. Basically its a concatenation of the type + as I want to search multiple classes of things within the same key. With longer IDs of 77 characters, I'm seeing that the records are never found with a NEARBY search.

When In invert the ID, (category at the end), which puts greater uniqueness up front, the record is found so I'm pretty sure that the issue is related to ID character length. Unrelated, I am trying to use "SEARCH" to debug but it always returns 0 count and IDs so I have to use SCAN with a high limit. This is version 1.4.1

[tidwall]

Are you using HTTP for the client?

[tidwall]

Or rather which client are you using? :)

[owaaa]

I'm using python redis client

[owaaa]

Thanks for the response. Also to be clear this is the Tile38 ID underneath the key, vs the redis key. Documentation on redis keys seems to be imply the ID can be fairly large. Is there an efficient way to use Tile38 to search across multiple keys? That would be an potential alternative approach to what I was trying to do but I didn't see any way to do that in the commands. Somewhat related, do I have to think about the redis max key size of 512MB and break apart my data or are thing implemented as aggregate types? My data is not yet that large but I plan it to be

[tidwall]

I'm not seeing this issue on my side. I created some test that use 5000+ length keys and ids with the '+' character separating every 16 bytes.

I'm wondering if this issue is related to #49? Perhaps the Python client or the network is fragmenting the packets and sending as smaller chunks where the boundaries land in the middle of a key or id.

could you verify again the recent push I made for #49?

Is there an efficient way to use Tile38 to search across multiple keys?

No, each key is it's own spatial index and is isolated from the other keys.

Do I have to think about the redis max key size of 512MB and break apart my data?

A key size has no limit, but each GeoJSON cannot be larger that 512MB. This is hardcoded and I can raise it you would like. It's really only there to keep bad clients from causing huge allocations on the server.

[owaaa]

Testing with master I saw inconsistencies once, but after flushing and trying all again I think its better. It may very well have been related to #49. Feel free to mark this as closed.

I still cannot get search to work however. Should I open a new ticket for that? I am testing over http I always get 0 results.

[tidwall]

The SEARCH command only works on the string type. It allows for managing basic sorted string data in a key. Geo types will not be returned with search.

For example:

SET trucks 0001:driver STRING Tom
SET trucks 0002:driver STRING Janet
SET trucks 0003:driver STRING Alexis
SET trucks 0004:driver STRING Aaron

SEARCH trucks 

Output:
"0004:driver", "Aaron"
"0003:driver", "Alexis"
"0002:driver", "Janet"
"0001:driver", "Tom"

SEARCH trucks MATCH A*

Output:
"0004:driver", "Aaron"
"0003:driver", "Alexis"

Sorry about the confusion. It appears that the SEARCH command is poorly documented on the tile38.com site. I'll fix this asap.

[owaaa]

ahh got it. I was trying to find some way to do a MATCH against a pattern of my IDs in some key (Without doing a geospatial search). SCAN with MATCH of a pattern != * didn't seem to work either.

[tidwall]

Could you provide an example of a pattern that you are using?

I generally use MATCH for something like this:

SET trucks 0001:pos POINT 33 -115
SET trucks 0001:dest POINT 43 -125
SET trucks 0002:pos POINT 34 -116
SET trucks 0002:dest POINT 44 -126
SET trucks 0003:pos POINT 35 -117
SET trucks 0003:dest POINT 45 -127

SCAN trucks MATCH 0001:* POINTS

Output:
"0001:dest", "43", "-125"
"0001:pos", "33", "-115"

[owaaa]

Nevermind. It is in fact working with SCAN on the master branch. I had another issue due to things I set up for the ID test. Thank you for the quick feedback on everything. Great Project

[tidwall]

Thank you for the helpful feedback, and please let me know if you have any further questions along the way.