[BUG] Document.get(id) returning None in app using FastAPI + Beanie

[agrin96]

Describe the bug I have a small web server that I am testing with a local mongodb instance. The app is FastAPI with beanie for the db operations. I've noticed that when I use id as a route parameter, calling document.get() returns None. However when I do a find().to_list() the results clearly show that the id is in fact in the DB.

To Reproduce

class User(Document):
    name: str
    package: Package
    token: str = Field(default_factory = lambda: secrets.token_hex(3))
    subscribed: datetime
    phone_number: constr(regex = r"^\d{3}-\d{3}-\d{4}$") | None
    address: str | None
    enabled: bool = True

    class Settings:
        name = "main"
        use_state_management = True
        bson_encoders = {
            datetime: lambda t: int(t.timestamp())
        }
        json_encoders = {
            ObjectId: str,
            datetime: lambda t: int(t.timestamp())
        }

    class Config:
        allow_population_by_field_name = True
        arbitrary_types_allowed = True
        json_encoders = {
            ObjectId: str,
            datetime: lambda t: int(t.timestamp())
        }
        schema_extra = {
            "example": {
                "name": "john smith",
                "subscribed": 1661123225,
                "phone_number": "917-323-2348",
                "address": "123 My apartment streat 32433"
            }
        }

@app.on_event('startup')
async def init_database():
    client = motor.motor_asyncio.AsyncIOMotorClient()
    await init_beanie(
        database = client.users,
        document_models = [User]
    )

@app.delete(
    "/users/{id}",
    response_description = "Delete a user from the system"
)
async def delete_user(id: PydanticObjectId):
    print(await User.find().to_list())

    print("search id: ", id)
    user = await User.get(document_id = id)
    print("user get: ", user) # prints None
    user: User | None = await User.find_one(User.id == id)
    print("user find_one: ", user) # prints None

Expected behavior What I expect to happen is that user = await User.get(document_id = id) returns the user object.

Additional context

This is the server output when using thunder client to hit the delete endpoint. As you can see the find_all returns a User that has 6321a13c270f743661ce9633 as an id. But when I call get() nothing is found.

Request example I used: http://127.0.0.1:8000/users/6321a13c270f743661ce9633

INFO:     Application startup complete.
[User(id=ObjectId('6321a12a270f743661ce9631'), revision_id=None, name='test user 1', package=skazka, token='de3d85', subscribed=datetime.datetime(2022, 10, 14, 20, 9, 34, tzinfo=datetime.timezone.utc), phone_number=None, address='test address', enabled=True), User(id=ObjectId('6321a132270f743661ce9632'), revision_id=None, name='test user 1', package=skazka, token='168409', subscribed=datetime.datetime(2022, 10, 14, 20, 9, 34, tzinfo=datetime.timezone.utc), phone_number=None, address='test address in some place', enabled=True), User(id=ObjectId('6321a13c270f743661ce9633'), revision_id=None, name='test user 1', package=skazka, token='da9c5b', subscribed=datetime.datetime(2022, 10, 17, 0, 12, 21), phone_number=None, address='test address 1', enabled=False), User(id=ObjectId('6321a293270f743661ce9634'), revision_id=None, name='test user 2', package=skazka, token='15f96d', subscribed=datetime.datetime(2022, 12, 14, 17, 24, 47, tzinfo=datetime.timezone.utc), phone_number=None, address=None, enabled=True), User(id=ObjectId('6321a8b95b34219c950bef31'), revision_id=None, name='test user 3', package=story, token='b00122', subscribed=datetime.datetime(2022, 12, 14, 17, 50, 53, tzinfo=datetime.timezone.utc), phone_number=None, address='some fake address', enabled=True), User(id=ObjectId('6321ddbdf776b6dc52ade020'), revision_id=None, name='Test user 4', package=skazka, token='0eb37f', subscribed=datetime.datetime(2022, 9, 22, 13, 56, 51, tzinfo=datetime.timezone.utc), phone_number='347-334-2343', address='some fake address', enabled=True), User(id=ObjectId('63220822bf998bced3a9fef5'), revision_id=None, name='Test user 5', package=skazka, token='1d2ea2', subscribed=datetime.datetime(2022, 10, 15, 3, 23, 59, tzinfo=datetime.timezone.utc), phone_number=None, address='other fake address', enabled=True), User(id=ObjectId('63220a3077e057a931350cf5'), revision_id=None, name='Aleksandr Grin', package=skazka, token='dc17f8', subscribed=datetime.datetime(2022, 9, 18, 17, 6, 49, tzinfo=datetime.timezone.utc), phone_number='917-841-6026', address='blah blah balh', enabled=True), User(id=ObjectId('63220ad2ac45aae56487668c'), revision_id=None, name='Alina ', package=skazka, token='c177e6', subscribed=datetime.datetime(2022, 9, 17, 17, 9, 29, tzinfo=datetime.timezone.utc), phone_number=None, address='woof', enabled=True)]
search id:  6321a13c270f743661ce9633
user get:  None
user find_one:  None

[agrin96]

Of course I spent several hours trying to figure this out and after I posted a bug report I figured it out. Let this be a warning for anyone who works with motor + pydantic and then switches to beanie + pydantic.

Somehow, the object ids created by the two are different even though they look exactly the same. I discovered that any users I created using my old motor method did not show up when using beanie.get() or find_one. I don't fully understand why, but after manually deleting the motor users everything works.

Hope this helps someone.

[roman-right]

Hey! Thank you for the issue. I'll check why ids are different. It is at least interesting. Or even a bug. Could you pls tell which MongoDB version you use?

[agrin96]

Hey, It was really interesting, minus the several hours of headache 😅 . I am using mongodb 4.2.3 which is not very recent. The guide I used originally to set up pydantic + mongo was from here . I suspect the issue may be with the PyObjectId class used somehow creating a different ID type, but I have no way to prove it since their string versions looked identical.