tortoise / tortoise-orm

Familiar asyncio ORM for python, built with relations in mind
https://tortoise.github.io
Apache License 2.0
4.68k stars 390 forks source link

Nested select_related raises KeyError inside _init_from_db #1396

Open Tauassar opened 1 year ago

Tauassar commented 1 year ago

Describe the bug When using nested select_related of form: 'modelA__modelB' I'm getting a KeyError when ORM tries to assign values to modelB.

An issue arises since kwargs values passed to _init_from_db function getting sliced version of model attribute's name, for example, instead of passing 'city_obj' attribute, 'city_ob' is being passed to kwargs, same way instead of attribute 'building', 'buildin' is being passed etc.

Actual error:

File "/usr/local/lib/python3.9/site-packages/tortoise/queryset.py", line 1008, in _execute instance_list = await self._db.executor_class( File "/usr/local/lib/python3.9/site-packages/tortoise/backends/base/executor.py", line 155, in execute_select obj = model._init_from_db( File "/usr/local/lib/python3.9/site-packages/tortoise/models.py", line 747, in _init_from_db setattr(self, key, meta.fields_map[key].to_python_value(value)) KeyError: 'buildin'

ORM version: tortoise-orm==0.19.3

jar3b commented 7 months ago

Same issue with 0.20.0

alexf-bond commented 2 months ago

Also facing this issue

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
app/common/decorators/timing.py:16: in wrap
    result = await f(*args, **kwargs)
app/api/ledger/get/get_financial_transactions.py:43: in process
    ) = await director.get_transactions_for_request(request=request)
app/common/decorators/timing.py:16: in wrap
    result = await f(*args, **kwargs)
app/common/builders/db/get_financial_transactions_director.py:39: in get_transactions_for_request
    return await self.get_results_for_request(request=request)
app/common/decorators/timing.py:16: in wrap
    result = await f(*args, **kwargs)
app/common/builders/db/get_double_entries_director_base.py:128: in get_results_for_request
    results, total_count = await self._builder.with_order(
app/common/decorators/timing.py:16: in wrap
    result = await f(*args, **kwargs)
app/common/builders/db/financial_transaction_query_builder.py:462: in results
    list_model = await model_creator.from_queryset(
/usr/local/lib/python3.11/site-packages/tortoise/contrib/pydantic/base.py:136: in from_queryset
    [submodel.model_validate(e) for e in await queryset.prefetch_related(*fetch_fields)]
/usr/local/lib/python3.11/site-packages/tortoise/queryset.py:1008: in _execute
    instance_list = await self._db.executor_class(
/usr/local/lib/python3.11/site-packages/tortoise/backends/base/executor.py:155: in execute_select
    obj = model._init_from_db(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'app.models.external_financial_account.ExternalFinancialAccount'>
kwargs = {'account_id': UUID('79b97392-e775-498c-85a1-d26268bcf012'), 'country': 'USA', 'currency': 'USD', 'date_created': datetime.datetime(2024, 9, 19, 16, 18, 57, 846057, tzinfo=datetime.timezone.utc), ...}
self = <ExternalFinancialAccount: <tortoise.fields.data.UUIDField object at 0x7f77fc6c4e50>>
meta = <tortoise.models.MetaInfo object at 0x7f77fc6cdf80>
key = 'mcc_description', model_field = 'brand_id'
field = <tortoise.fields.data.UUIDField object at 0x7f77fc6c4750>
value = "MEN'S AND WOMEN'S CLOTHING STORES"

    @classmethod
    def _init_from_db(cls: Type[MODEL], **kwargs: Any) -> MODEL:
        self = cls.__new__(cls)
        self._partial = False
        self._saved_in_db = True
        self._custom_generated_pk = self._meta.db_pk_column not in self._meta.generated_db_fields

        meta = self._meta

        try:
            # This is like so for performance reasons.
            #  We want to avoid conditionals and calling .to_python_value()
            # Native fields are fields that are already converted to/from python to DB type
            #  by the DB driver
            for key, model_field, field in meta.db_native_fields:
                setattr(self, model_field, kwargs[key])
            # Fields that don't override .to_python_value() are converted without a call
            #  as we already know what we will be doing.
            for key, model_field, field in meta.db_default_fields:
                value = kwargs[key]
                setattr(
                    self,
                    model_field,
                    None if value is None else field.field_type(value),
                )
            # These fields need manual .to_python_value()
            for key, model_field, field in meta.db_complex_fields:
                setattr(self, model_field, field.to_python_value(kwargs[key]))
        except KeyError:
            self._partial = True
            # TODO: Apply similar perf optimisation as above for partial
            for key, value in kwargs.items():
>               setattr(self, key, meta.fields_map[key].to_python_value(value))
E               KeyError: 'mcc_description'

What's odd is that it's intermittent and not the same model/property each time