[Bug] Find with $lte/$gte and datetime doesn't work in latest Beanie version 1.8.3

[UtkarshMish]

The issue is that Document with find, find_many, doesn't work and return empty List when using with $lte,$gte, operators with datetime fields.

[roman-right]

Hey @UtkarshMish ,

The next test is green: https://github.com/roman-right/beanie/blob/main/tests/odm/query/test_find.py#L305

Could you pls add a code example, to repeat this bug?

[UtkarshMish]

sure,

What i have done is tried to print all the data by applying the different GTE operator first, then using the == operator, and in last i have fetched all the data and applied filter to the particular datetime value.

I am getting the required solution only in the last one where i have applied filter. Please check below:

Code Snippet:

date_time = datetime(2021, 9, 28, 11, 51) print(date_time) print( await Drones.find_many( {"first_launch": {"$gte": date_time}} ).to_list() ) print( await Drones.find_many(GTE(Drones.first_launch, date_time)).to_list() ) print( await Drones.find_one(Drones.first_launch == date_time) ) print([*filter( lambda item:item.first_launch == date_time, await Drones.all().to_list())]) return await Drones.find_many(GTE(Drones.first_launch, date_time)).to_list()

The output for the following is:

[roman-right]

Hm, it looks weird.

Can you pls share the model code too?

I'm trying the same operations with the same data here and it works:

import asyncio
import datetime

import motor.motor_asyncio

from beanie import Document, init_beanie
from beanie.operators import GTE

class Sample(Document):
    ts: datetime.datetime

async def main():
    uri = "YOUR MONGODB DSN"
    cli = motor.motor_asyncio.AsyncIOMotorClient(uri)
    await init_beanie(cli.test_db,
                      document_models=[Sample])

    date_time = datetime.datetime(2021, 9, 28, 11, 51)

    await Sample(ts=date_time).insert()
    docs = await Sample.find(GTE(Sample.ts, date_time)).to_list()
    print(docs)

asyncio.run(main())

output:

[Sample(id=ObjectId('61b1fcfc7b3634562d5a2042'), revision_id=None, ts=datetime.datetime(2021, 9, 28, 11, 51))]

[UtkarshMish]

Okay, So I checked it once again and found that, it is a bug on version 1.8.3 specifically. Please check my requirements.txt also if necessary. If you use older beanie version like 1.8.0, it will work. (checked) So I have here , my code repo model: https://github.com/UtkarshMish/drone-inspector/blob/master/app/database/Drones.py

I have made an api for testing for the same. Link: https://drone-related-app.herokuapp.com/api/drones?date=2019-10-18T22%3A51%3A26.000

repo link: https://github.com/UtkarshMish/drone-inspector/

[roman-right]

It looks like I found a potential problem. Could you pls try with 1.8.4? Also, pls rewrite the data if possible. It could be inconsistent. By mistake, it stored datetime objects as strings instead of timestamps

[UtkarshMish]

Verified, Found everythin working in the latest patch. Thank you so much ! , You are awesome ~!

[roman-right]

Thank you for the catch! It is a critical one