redis / redis-om-python

Object mapping, and more, for Redis and Python
MIT License
1.12k stars 114 forks source link

Find operation on datetime field raises error --> argument of type 'datetime.datetime' is not iterable #199

Open msarm opened 2 years ago

msarm commented 2 years ago

Find operation on datetime field raises error --> argument of type 'datetime.datetime' is not iterable

import datetime
from typing import Optional

from redis_om import Field, HashModel, Migrator, get_redis_connection

# This Redis instance is tuned for durability.
REDIS_DATA_URL = "redis://localhost:6380"

class Person(HashModel):
    first_name: str = Field(index=True)
    last_name: str = Field(index=True)
    birth_datetime: datetime.datetime = Field(index=True)

# set redis connection
Person.Meta.database = get_redis_connection(url=REDIS_DATA_URL,
                                                  decode_responses=True)
# apply migrations
Migrator().run()

# First, we create a new `person` object:
person = Person(
    first_name="John",
    last_name="Smith",
    birth_datetime = datetime.datetime(2002, 4, 9, 9, 13)

)

# print globally unique primary key
print(person.pk)

# save person to Redis
person.save()

# get by primary key (WORKS!!!)
Person.get(person.pk)

# Find all person with first name "John" - WORKS!!
Person.find(Person.first_name == "John").all()

# find all person with birth date time with date as string - NOT WORKING!! ( Returns empty list )
Person.find(Person.birth_datetime == '2002-04-09 09:13').all()

# find all person with birth date time with date type - Not working!! ( Error: argument of type 'datetime.datetime' is not iterable )
Person.find(Person.birth_datetime == datetime.datetime(2002, 4, 9, 9, 13)).all()
simonprickett commented 2 years ago

Hi there - thanks for posting example code, I'll take a look at this and see what I can find.

simonprickett commented 2 years ago

Some debugging notes to myself...

Running MONITOR against the Redis instance I can see that this:

Person.find(Person.birth_datetime == '2002-04-09 09:13').all()

results in this RediSearch query:

"ft.search" ":__main__.Person:index" "@birth_datetime:{2002\\-04\\-09\\ 09\\:13}" "LIMIT" "0" "10"

I can also see that the datetime field was created as a tag field in the index:

"ft.create" ":__main__.Person:index" "ON" "HASH" "PREFIX" "1" ":__main__.Person:" "SCHEMA" "pk" "TAG" "SEPARATOR" "|" "first_name" "TAG" "SEPARATOR" "|" "last_name" "TAG" "SEPARATOR" "|" "birth_datetime" "TAG" "SEPARATOR" "|"

and that values for birth_datetime are stored as a date string:

127.0.0.1:6379> HGETALL :__main__.Person:01G0H6YETC4WSAS668QHDERZDB
1) "pk"
2) "01G0H6YETC4WSAS668QHDERZDB"
3) "first_name"
4) "John"
5) "last_name"
6) "Smith"
7) "birth_datetime"
8) "2002-04-09T09:13:00"

This will work for exact matches, but probably should be more intuitive or at least documented and may vary depending on your locale settings:

Person.find(Person.birth_datetime == '2002-04-09T09:13:00').all()

Ideally too the way date/time is indexed could change so that it's possible to do range queries over it (probably need to store the date/time as a timestamp for this).

msarm commented 2 years ago

@simonprickett - That's a nice finding, it stores as a date string based on the locale settings.

Yes, as you called out for any date range queries it is good to go with the timestamp.