[BUG] Backlinks are not populated

[sheoak]

Describe the bug When creating backlinks, the documentation (to my knowledge) doesn’t show a way to retrieve them immediately. This has been asked several times in the questions section, but I realized none of them are answered so I post this as an issue now.

Is there a way to populate the backlink, similar to fetch_all_links with Link objects? (see example below)

To Reproduce Considering the definitions below:

class IdCard(Document):
    ref: Indexed(str, unique=True)
    # Only one owner per id card
    owner: BackLink["Person"] | None = Field(original_field="id_card")

class Person(Document):    
    name: str
    parents: List[Link["Person"]] | None = None
    kids: List[BackLink["Person"]] | None = Field(original_field="parents") 

morticia = Person(name="Morticia")
gomez = Person(name="Gomez")
await morticia.insert()
await gomez.insert()

wednesday = Person(name="Wednesday", parents=[gomez, morticia])
await wednesday.insert()

The following does NOT work:

# If uncommented, AttributeError: 'BackLink' object has no attribute 'id'
# await morticia.fetch_all_links()
# AttributeError: 'BackLink' object has no attribute 'name'
print(morticia.kids[0].name)

This works:

result = await Person.find(Person.name == 'Morticia', fetch_links=True).first_or_none()
print(result.kids[0].name)

Expected behavior Backlinks should be populated

[roman-right]

Hi @sheoak , thank you for the issue. I'll check what is going on there and will update you here. It is probably a bug

[bedlamzd]

Looked at the implementation - fetch_all_links doesn't implement fetching BackLinks. Maybe @roman-right accidently forgot it.

In this case the following happens:

1. Document.fetch_all_links calls Document.fetch_link for each link
2. This branch in Document.fetch_link is executed, where ref_obj is of type list[BackLink]

       if isinstance(ref_obj, list) and ref_obj:
           values = await Link.fetch_list(ref_obj, fetch_links=True)
           setattr(self, field, values)

   Note that Link.fetch_list is called

3. it calls Link.repack_links to get linked documents that aren't fetched yet
4. which tries to get id from the link here and fails, because BackLink doesn't have id

So at the moment you have to use find methods

[renja-g]

So at the moment you have to use find methods

I'm not quite sure if this is related, but when I try to find documents that have a BackLink I get an error, that the BackLink field can not be encoded If you have any ideas how to work around this, please tell me 😉

Error

``` Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/home/renja/projects/leaderboard/backend/app/test.py", line 52, in asyncio.run(init()) File "/home/renja/.pyenv/versions/3.12.2/lib/python3.12/asyncio/runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/home/renja/.pyenv/versions/3.12.2/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/renja/.pyenv/versions/3.12.2/lib/python3.12/asyncio/base_events.py", line 685, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/app/test.py", line 45, in init league_entries = await LeagueEntry.find(LeagueEntry.summoner == summoner).to_list() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/queries/cursor.py", line 72, in to_list cursor = self.motor_cursor ^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/queries/find.py", line 688, in motor_cursor filter=self.get_filter_query(), ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/queries/find.py", line 106, in get_filter_query return Encoder(custom_encoders=self.encoders).encode( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/utils/encoder.py", line 134, in encode return {str(key): self.encode(value) for key, value in obj.items()} ^^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/utils/encoder.py", line 127, in encode return self._encode_document(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/utils/encoder.py", line 110, in _encode_document obj_dict[key] = sub_encoder.encode(value) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/utils/encoder.py", line 136, in encode return [self.encode(value) for value in obj] ^^^^^^^^^^^^^^^^^^ File "/home/renja/projects/leaderboard/backend/.venv/lib/python3.12/site-packages/beanie/odm/utils/encoder.py", line 138, in encode raise ValueError(f"Cannot encode {obj!r}") ValueError: Cannot encode ```

Code

```py import asyncio from beanie import init_beanie, Document from motor.motor_asyncio import AsyncIOMotorClient from app.models import Summoner, LeagueEntry async def init(): client = AsyncIOMotorClient( "mongodb://localhost:27017", ) await init_beanie(database=client.test_db, document_models=[Summoner, LeagueEntry]) summoner = Summoner( game_name = "test", name = "test", platform = "test", profile_icon_id = 1, puuid = "test", summoner_id = "test", summoner_level = 1, tag_line = "test", ) await summoner.insert() league_entry = LeagueEntry( league_id="test", queue_type="test", tier="test", rank="test", league_points=1, wins=1, losses=1, veteran=False, inactive=False, fresh_blood=False, hot_streak=False, summoner=summoner, ) await league_entry.insert() # get all league entries of the summoner league_entries = await LeagueEntry.find(LeagueEntry.summoner == summoner).to_list() print(league_entries) asyncio.run(init()) ```

Models

```py import pymongo from pydantic import BaseModel, Field from beanie import Document, Link, BackLink # A summoner can have multiple league entries or none at all # A league entry is linked to a summoner class Summoner(Document): game_name: str name: str platform: str profile_icon_id: int puuid: str summoner_id: str summoner_level: int tag_line: str league_entries: list[BackLink["LeagueEntry"]] = Field(original_field="summoner") class Settings: name = "summoner" indexes = [ [ ("game_name", pymongo.TEXT), ("tag_line", pymongo.TEXT), ("platform", pymongo.TEXT), ] ] class LeagueEntry(Document): league_id: str queue_type: str tier: str rank: str league_points: int wins: int losses: int veteran: bool inactive: bool fresh_blood: bool hot_streak: bool summoner: Link[Summoner] class Settings: name = "league_entry" indexes = [ [ ("tier", pymongo.TEXT), ("rank", pymongo.TEXT), ("league_points", pymongo.TEXT), ] ] ```

[valentinoli]

I don't know if this is related, but in Pydantic v2 it seems like the way to define a BackLinks original_field is different from the description in the docs. From reading the source code the following seems to be the way to do it:

class Door(Document):
    house: BackLink[House] = Field(json_schema_extra={"original_field": "door"})

as opposed to

class Door(Document):
    house: BackLink[House] = Field(original_field="door")

[sheoak]

I don't know if this is related, but in Pydantic v2 it seems like the way to define a BackLinks original_field is different from the description in the docs

Yes, and the new format is poorly documented. But it still doesn’t work in my case.

[cvasilatos]

Any updates on this?

[cutesweetpudding]

Any updates on this? Is there a way on demand fetch one particular BackLink field? Can someone share an example?

[ltieman]

Yeah, this is causing issues for us as well, the weird part is that it seems to periodically work. Has any work been done on this?

[ltieman]

it also seems that there are a lot of tickets on nested links not populating that have been closed by the bot even though they have not been completed. on the links we can run through the object and force them to populate, but because it isnt really possible to fetch a backlink after the initial search, this becomes quite difficult

[rosjat]

Will this be resolved in the near future ?

Beside that it's a lot of try and error to get the magic behind it ... the documentation is kinda minimal when it comes to stuff like that. At the moment the whole code we use isn't even remotely populating something on the list of backlinked documents.