[BUG] ElemMatch on Document property of Type List[Link] fails with IndexError in relations.py convert_ids() beanie==1.15.4

[TLeitzbach]

First of all, thank you so much for the great package!!!

Describe the bug beanie fails to properly convert an ElemMatch Query on a Document property of Type Optional[List[Link]] and raises 'beanie\odm\utils\relations.py", line 65, in convert_ids and k.split(".")[1] == "id" IndexError: list index out of range'

To Reproduce Python 3.9

requirements.txt beanie==1.15.4 pydantic==1.9.2 pymongo[srv]==4.1.1

from typing import Optional
from beanie import Document, Link
from beanie.operators import ElemMatch

import asyncio

class DocToLink(Document):

    class Settings:
        name = 'LinkExample'

    child_name: str

class DocWithLinkAttribute(Document):

    class Settings:
        name = 'ParentExample'

    parent_name: str
    linked_docs: Optional[list[Link[DocToLink]]]

async def add_and_query(child_name: str, parent_name: str) -> Optional[list[DocWithLinkAttribute]]:
    """"""

    child: DocToLink = DocToLink(child_name=child_name)
    await child.insert()

    parent: DocWithLinkAttribute = DocWithLinkAttribute(parent_name=parent_name, linked_docs=[child])
    await parent.insert()

    queried_doc:  list[DocWithLinkAttribute] = await DocWithLinkAttribute.find(
            ElemMatch(DocWithLinkAttribute.linked_docs, DocToLink.child_name == child_name), fetch_links=True
            ).to_list()
    return queried_doc

async def init_mongo_client(models: list[Document]):
    """"""
    mongoClient = mdb(secretName=secretMONGO)
    await init_beanie(database=mongoClient.client[mongoClient.databaseInfo["database"]],
                      document_models=models)
    mongoClient.client.get_io_loop = asyncio.get_running_loop

models: list = [DocWithLinkAttribute, DocToLink]
asyncio.run(init_mongo_client(models))

docs = asyncio.run(add_and_query("I'm a child", "I'm the parent"))

Traceback (most recent call last): File "", line 55, in File "Python\Python39\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "Python\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete return future.result() File "", line 35, in add_and_query File "\venv\lib\site-packages\beanie\odm\queries\cursor.py", line 71, in to_list cursor = self.motor_cursor File "\venv\lib\site-packages\beanie\odm\queries\find.py", line 609, in motor_cursor aggregation_pipeline.append({"$match": self.get_filter_query()}) File "\venv\lib\site-packages\beanie\odm\queries\find.py", line 105, in get_filter_query self.prepare_find_expressions() File "\venv\lib\site-packages\beanie\odm\queries\find.py", line 93, in prepare_find_expressions self.find_expressions[i] = convert_ids( File "\venv\lib\site-packages\beanie\odm\utils\relations.py", line 65, in convert_ids and k.split(".")[1] == "id" IndexError: list index out of range

Expected behavior The query (debugger shows: {'linked_docs': {'$elemMatch': {'child_name': "I'm a child"}}}) should return the correct (list) of documents. In, fact if ' and k.split(".")[1] == "id"' is commented out and new_k is set to k as in line 72 of relations.py, the output is given correctly. E.g.

  for k, v in query.items():
      if (
          isinstance(k, ExpressionField)
          and doc.get_link_fields() is not None
          and k.split(".")[0] in doc.get_link_fields().keys()  # type: ignore
          # and k.split(".")[1] == "id"
      ):
          if fetch_links:
              new_k = k
              # new_k = f"{k.split('.')[0]}._id"
          else:
              new_k = f"{k.split('.')[0]}.$id"
      else:
          new_k = k

Unfortunately, however the error prevents this from happening. I am not sure where the "id" should come from or how to fix it properly as of now.

I have seen the "# TODO add all the cases" in "convert_ids()" but I was not aware whether this came to your mind already.

Additional context Thank you very much! I really appreciate the package. Let me know if I can help you on this:) Best, Thilo

[Abashinos]

Hey @TLeitzbach, We're also facing this behavior. Notably, the whole query seems to work if the In subquery is wrapped in another operator like this:

q = Or(In(Model.a, ...), In(Model.b, ...))

Are you aware about the latest version where this issue is not present? I might consider rolling back to an earlier version until this bug is patched

[Abashinos]

I've tried a number of versions from 1.12 up to 1.15, all seem to display this behavior. Plain mongo query works though:

{"a": {"$in": ...}}

[roman-right]

Hey @TLeitzbach , Please try 1.16.2

[TLeitzbach]

Hey @roman-right, super cool! Will check it out tomorrow:) Thanks!!!

[TLeitzbach]

Tested in 1.16.4 and worked out fine:) Cheers!!!

[Abashinos]

I can confirm the issue is fixed now. Thanks a lot, @roman-right !