BeanieODM / beanie

Asynchronous Python ODM for MongoDB
http://beanie-odm.dev/
Apache License 2.0
1.91k stars 201 forks source link

[BUG] Inheritance causes conflict index creation if children have different TTL indexes #416

Closed ssttkkl closed 1 year ago

ssttkkl commented 1 year ago

Sorry for my poor English.

Describe the bug Inheritance causes conflict index creation if children have different TTL indexes

To Reproduce

import asyncio
from datetime import datetime
from typing import Any

from beanie import Document, init_beanie
from motor.motor_asyncio import AsyncIOMotorClient
from pymongo import IndexModel

class Cache(Document):
    key: str
    value: Any
    update_time: datetime = datetime.utcnow()

class SearchCache(Cache):
    class Settings:
        name = "search_cache"
        indexes = [
            IndexModel([("update_time", 1)], expireAfterSeconds=3600)
        ]

class DetailCache(Cache):
    class Settings:
        name = "detail_cache"
        indexes = [
            IndexModel([("update_time", 1)], expireAfterSeconds=3600 * 2)
        ]

class ImageCache(Cache):
    class Settings:
        name = "image_cache"
        indexes = [
            IndexModel([("update_time", 1)], expireAfterSeconds=3600 * 3)
        ]

async def main():
    client = AsyncIOMotorClient("mongodb://localhost:27017")
    await init_beanie(database=client.test_db, document_models=[SearchCache, DetailCache, ImageCache])

    cache = SearchCache(key="search", value="test2")
    await cache.save()

    cache = DetailCache(key="detail", value="test2")
    await cache.save()

    cache = ImageCache(key="image", value="test3")
    await cache.save()

    client.close()

if __name__ == "__main__":
    asyncio.run(main())

These two collections were created:

> show collections
< Cache
  search_cache

Then it crashed:

Traceback (most recent call last):
  File "C:\Users\huang\PycharmProjects\pythonProject\main.py", line 57, in <module>
    asyncio.run(main())
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 641, in run_until_complete
    return future.result()
  File "C:\Users\huang\PycharmProjects\pythonProject\main.py", line 42, in main
    await init_beanie(database=client.test_db, document_models=[SearchCache, DetailCache, ImageCache])
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\beanie\odm\utils\init.py", line 456, in init_beanie  
    await Initializer(
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\beanie\odm\utils\init.py", line 89, in __await__     
    yield from self.init_class(model).__await__()
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\beanie\odm\utils\init.py", line 426, in init_class   
    await self.init_document(cls)
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\beanie\odm\utils\init.py", line 328, in init_document
    await self.init_indexes(cls, self.allow_index_dropping)
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\beanie\odm\utils\init.py", line 281, in init_indexes 
    new_indexes += await collection.create_indexes(found_indexes)
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\pymongo\collection.py", line 1865, in create_indexes
    return self.__create_indexes(indexes, session, **kwargs)
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\pymongo\collection.py", line 1900, in __create_indexes
    self._command(
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\pymongo\collection.py", line 272, in _command
    return sock_info.command(
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\pymongo\pool.py", line 743, in command
    return command(
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\pymongo\network.py", line 160, in command
    helpers._check_command_response(
  File "C:\Users\huang\AppData\Local\Programs\Python\Python310\lib\site-packages\pymongo\helpers.py", line 180, in _check_command_response
    raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { update_time: 1 }, name: "update_time_1", expireAfterSeconds: 10800 }, existing index: { v: 2, key: { up
date_time: 1 }, name: "update_time_1", expireAfterSeconds: 7200 }, full error: {'ok': 0.0, 'errmsg': 'An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { update_time: 1 }, name: "update_ti
me_1", expireAfterSeconds: 10800 }, existing index: { v: 2, key: { update_time: 1 }, name: "update_time_1", expireAfterSeconds: 7200 }', 'code': 85, 'codeName': 'IndexOptionsConflict'}

Expected behavior Cache collection is not expected (but actually it was created, I don't know if it's by design). There should be three collections: search_cache, detail_cache, and image_cache, with different TTL indexes.

Additional context It works on v1.31.1, but fails on v1.15.3.

Vitalium commented 1 year ago

I think in that case it is better to subclass from pydantic.BaseModel rather than beanie.Document: I.e. this

class Cache(BaseModel):
    ...

instead of

class Cache(Document):
    ...