redis / redis-om-dotnet

Object mapping, and more, for Redis and .NET
MIT License
457 stars 76 forks source link

Time out does not specified in slow query #334

Closed imansafari1991 closed 1 year ago

imansafari1991 commented 1 year ago

Hi there, I had an issue cause of my query latency, but I didn't understand before spending much time. I've tried to execute the following query:

          var checkInUnixTime = checkIn.Value.ToUnixTimeSeconds();
                var checkOutUnixTime = checkOut.Value.ToUnixTimeSeconds();
                roomDailySpecificAggregations =
                    roomDailySpecificAggregations.Filter(x => x.RecordShell.DateTimeLong < checkOutUnixTime);
                roomDailySpecificAggregations =
                    roomDailySpecificAggregations.Filter(x => x.RecordShell.DateTimeLong >= checkInUnixTime);
                roomDailySpecificAggregations =
                    roomDailySpecificAggregations.Filter(x => x.RecordShell.IsBlocked < 1);
                roomDailySpecificAggregations = roomDailySpecificAggregations
                        .GroupBy(p => new {p.RecordShell.RoomId})
                        .CountGroupMembers()

                        .Average(z => z.RecordShell.AfterDiscountAmount)
                        .Filter(p => p["AfterDiscountAmount_AVG"] <= maxPrice)
                        .Filter(p => p["AfterDiscountAmount_AVG"] >= minPrice)
                        .Filter(p => p.Aggregations["COUNT"] == stayingDays);

              var  roomAggregationList = roomDailySpecificAggregations.ToListAsync();

But sometimes, according to the database's existing keys, I've got an "empty" list, while many keys have the above condition. After searching about this issue, I found some articles about "query time out" in the redis search. https://redis.io/docs/stack/search/configuring/

ON_TIMEOUT

The response policy for queries that exceed the TIMEOUT setting.

The policy can be one of the following:

RETURN: this policy will return the top results accumulated by the query until it timed out. FAIL: will return an error when the query exceeds the timeout value.

Default

RETURN

After changing the policy to FAIL, I expected to get an Exception in redis.om , but I didn't.

In my opinion, if the error is displayed, it can help clarify the case more.

I tried to add an exception in this situation I could not go anywhere other than https://github.com/redis/redis-om-dotnet/blob/main/src/Redis.OM/RedisReply.cs

Isn't it better to throw an exception user in this situation?

Thanks

slorello89 commented 1 year ago

Hmm @imansafari1991 - it appears that the "failure" isn't being reported as an error by Redis see this output from telnet (I have a a ufo sightings database that I use for stress testing):

set foo bar
+OK
incr foo
-ERR value is not an integer or out of range
FT.SEARCH sightings *
*1
+Timeout limit was reached
FT.SEARCH sightings * LIMIT 0 0
*1
+Timeout limit was reached

at a protocol level - Redis isn't responding with the traditional - to tell you it's an error, so StackExchange.Redis - doesn't throw anything because as far as they are concerned, the command succeeded, will need to add a special branch to check this.

slorello89 commented 1 year ago

But you're totally right - I would have 100% expected this to just throw an error.

imansafari1991 commented 1 year ago

Thank you @slorello89 Should we open an issue on StackExchange.Redis for more investigation?

slorello89 commented 1 year ago

No - it's a problem with RediSearch, not StackExchange.Redis, They're just replying with what they think is a valid result (which it claims to be) - I've raised this to our engineering team, they are going to look at fixing it, in the interim, it's up Redis OM to try to detect this condition and handle it appropriately.

VagyokC4 commented 1 year ago

Sounds like the issue I was having here maybe => #262 In my case maybe it was better that it returned some results and not thrown an error, but it def. should be more apparent what is happening, as I didn't notice it until I had a large dataset I was working with.

imansafari1991 commented 1 year ago

It happens when there is no aggregation in query, but as you said If we have aggregation query on large data does not return anything.