JSON.QGET doesn't work on non-String fields

[fstarr]

Hi,

thanks for providing the community with the RedisJSON2 extension!

TL;DR

The indexing (or querying) mechanism of RedisJSON2 does not seem to work for non-string fields.

Could this be fixed, so that JSON.QGET also allows queries involving non-string fields?

Detailed description (including reproducer) follows

I'm creating an index users as follows and insert two data sets:

1573724213.769257 [0 127.0.0.1:58322] "SELECT" "7"
1573724213.769399 [7 127.0.0.1:58322] "FLUSHDB"

1573724584.246502 [7 127.0.0.1:58322] "JSON.INDEX" "ADD" "users" "first" "$.first"
1573724584.246800 [7 127.0.0.1:58322] "JSON.INDEX" "ADD" "users" "last" "$.last"
1573724584.247056 [7 127.0.0.1:58322] "JSON.INDEX" "ADD" "users" "age" "$.age"

1573724648.064295 [7 127.0.0.1:60454] "JSON.SET" "user1" "." "{\"first\": \"Joe\", \"last\": \"Smith\"}" "INDEX" "users"
1573724651.026177 [7 127.0.0.1:60454] "JSON.SET" "user2" "." "{\"first\": \"Kevin\", \"last\": \"Smith\", \"age\": 33}" "INDEX" "users"

Next I'm executing a query asking for all users whose last names start with "smi":

1573724379.240656 [7 127.0.0.1:58322] "JSON.QGET" "users" "@last:smi*"

The query returns all users as expected:

{"user1":[{"first":"Joe","last":"Smith"}],"user2":[{"first":"Kevin","last":"Smith","age":33}]}

However when asking for users with an age of 33:

1573724848.397281 [7 127.0.0.1:60454] "JSON.QGET" "users" "@age:33"

Nothing is returned:

{}

If I insert a third user with an age specified as string (note the difference to the user2 insertion):

1573725060.306817 [7 127.0.0.1:60454] "JSON.SET" "user3" "." "{\"first\": \"Some\", \"last\": \"Dude\", \"age\": \"33\"}" "INDEX" "users"

And rerun the query:

1573725213.414359 [7 127.0.0.1:60454] "JSON.QGET" "users" "@age:33"

The data set for user3 is returned.

{"user3":[{"first":"Some","last":"Dude","age":"33"}]}

This observation leads to the conclusion that the indexing (or querying) mechanism of RedisJSON2 does not work for non-string fields.

Could this be fixed, so that JSON.QGET also allows queries involving non-string fields, like the one I created for user2?

Following you'll find my reproducer script that was hacked together in Python. I used redis-cli monitor to monitor the actual commands hitting redis as shown above.

import redis
import json

redis_host = "localhost"
redis_db = 7
r = redis.StrictRedis(host=redis_host, db=redis_db)

r.flushdb()

# %%

index = 'users'

try:
    r.execute_command('JSON.INDEX', 'ADD', index, 'first', '$.first')
except:
    print("Index probably exists")
try:
    r.execute_command('JSON.INDEX', 'ADD', index, 'last', '$.last')
except:
    print("Index probably exists")
try:
    r.execute_command('JSON.INDEX', 'ADD', index, 'age', '$.age')
except:
    print("Index probably exists")

# %%

data_1 = {
    'first': 'Joe',
    'last': 'Smith'
}

r.execute_command('JSON.SET', 'user1', '.', json.dumps(data_1), 'INDEX', index)

# %%

data_2 = {
    'first': 'Kevin',
    'last': 'Smith',
    'age': 33
}

r.execute_command('JSON.SET', 'user2', '.', json.dumps(data_2), 'INDEX', index)

# %%

print(r.execute_command('JSON.QGET', index, '@last:smi*'))

# %%

print(r.execute_command('JSON.QGET', index, '@age:33'))

# %%

data_3 = {
    'first': 'Some',
    'last': 'Dude',
    'age': '33'
}

r.execute_command('JSON.SET', 'user3', '.', json.dumps(data_3), 'INDEX', index)

# %%

print(r.execute_command('JSON.QGET', index, '@age:33'))

[gkorland]

Thanks for the issue!

[gkorland]

@fstarr the right way to query numeric fields is using range queries in your example it should be:

127.0.0.1:6379> "JSON.QGET" "users" "@age:[33,33]"
"{\"user2\":[{\"first\":\"Kevin\",\"last\":\"Smith\",\"age\":33}]}"

The query language is following RediSearch query syntax See: https://oss.redislabs.com/redisearch/Query_Syntax.html

[fstarr]

@gkorland Thanks! For integer values this works as expected! But unfortunately it does not work for float numbers. :(

Changing the age of the example dataset above to 33.1 and rerunning the query again yields no results:

127.0.0.1:6379> "JSON.QGET" "users" "@age:[33.1,33.1]"
"{}"

Might be a floating point number comparison problem that probably is unrelated to RedisJSON2:

127.0.0.1:6379> "JSON.QGET" "users" "@age:[33.0,33.2]"
"{\"user2\":[{\"first\":\"Kevin\",\"last\":\"Smith\",\"age\":33.1}]}"

[cw1427]

Hi,

May I have a question that: does redisJSON2 runs depends on redisSearch? I mean we do need to also prepare loading redisSearch so lib ?

[gkorland]

Yes it does depends on RediSearch. But, no need to also load load RediSearch so, it packs RediSearch logic as part of RedisJSON2 so.

[gatomazi]

Hello,

May I have another question: Is it possible to query a field that is inside another obj? Like this:

172.17.0.1:7001> "JSON.SET" "user4" "." "{\"first\": \"Will\", \"last\": \"Smith\", \"location\": {\"city\": \"Florianopolis\", \"uf\": \"SC\"}}" "INDEX" "users"

I want to find every location.uf = SC. Like a "@location.uf:SC"

[gkorland]

Sure you should add an index on location.uf and then you can query.

JSON.INDEX add users uf $.location.uf

JSON.QGET users "@uf:SC" 

[gkorland]

BTW for questions you might want to use the group https://groups.google.com/forum/#!forum/redisjson

[fstarr]

@gkorland I saw you're quite active over at RediSearch. Do you happen to know if this issue regarding floating point range queries is already known there? If not, would it be ok to repost the issue there (with the above RedisJSON2 reproducer)? Maybe there's a way to link the discussion of this thread too.

[gkorland]

We are already discussing it on RediSearch. I just opened this issue the other day https://github.com/RediSearch/RediSearch/issues/1080