KeyError xxx :: The program was running normally when this error suddenly appeared

[ChangeMoreNate]

Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/tortoise/models.py", line 708, in _init_from_db setattr(self, model_field, kwargs[key]) KeyError: 'account'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data/hongliaoback/frontend/api.py", line 2181, in getUserInfo apply_obj = await AuthorApply.get(user_id=user_obj.id) File "/usr/local/lib/python3.8/dist-packages/tortoise/queryset.py", line 871, in _execute instance_list = await self._db.executor_class( File "/usr/local/lib/python3.8/dist-packages/tortoise/backends/base/executor.py", line 132, in execute_select instance: "Model" = self.model._init_from_db( File "/usr/local/lib/python3.8/dist-packages/tortoise/models.py", line 721, in _init_from_db setattr(self, key, meta.fields_map[key].to_python_value(value)) KeyError: 'username'

[mocurin]

Same issue with 0.17.6

I believe you can reproduce this issue by creating asyncio tasks with tortoise entities created inside transactions (in my case entity was created with select_for_update). If you modify & save such entity inside task - it works fine, but other operations fail after receiving this error: pymysql.err.InternalError: Packet sequence number wrong - got 105 expected 1

Further db calls fail with key errors on entities initialization:

  File "/usr/local/lib/python3.8/site-packages/tortoise/queryset.py", line 890, in _execute
    instance_list = await self._db.executor_class(
  File "/usr/local/lib/python3.8/site-packages/tortoise/backends/base/executor.py", line 132, in execute_select
    instance: "Model" = self.model._init_from_db(
  File "/usr/local/lib/python3.8/site-packages/tortoise/models.py", line 730, in _init_from_db
    setattr(self, key, meta.fields_map[key].to_python_value(value))
KeyError: 'name'

Which seems like database connector ended up broken somehow Also "packet sequence" error may be silent, but the outcome is pretty much the same

Ill look into this issue later. It seems to be pretty hard to reproduce, but its here and i'm pretty sure that something either in transactions, or in my usage of them is very wrong. In the latter case there at least should be some useful error.

[ChangeMoreNate]

@mocurin Thank you very much for your help, I am currently working on a solution by increasing the number of thread pool connections to mysql. In the meantime I am also investigating the cause of the code error and I will start with what you have said.

[mocurin]

Thanks, @ChangeMoreNate !

Until then there are some steps which helped me to avoid this issue:

1. Make transactions shorter - commit as soon as you are done and do not 'loose connection' to the task you created with entity in transaction. If you can't avoid it - at least wait until its completion in the issuer coroutine.
2. Take a look at something other that transactions - in my case the solution was to add some flag and/or state enum field in the model. Then you should filter entities by these fields. At first it seems that it is too much work to maintain and check these flags, but i believe that it's the only solution when fields updates depend on something that takes several minutes to complete (although you really should consider avoiding such interactions, especially inside transactions).
3. I'm not really sure, but i believe you can catch these errors and restore connector. This would require inheriting from tortoise Model-class and wrapping create-classmethod. But its better to do something with app architecture and make transactions shorter somehow.

All in all, this error is extremely specific and happens in rare cases. And the issue is definitely in the wrong transaction usage and probably in bad architecture. It would be great if Tortoise documentation explained something more, than just transaction usage - some hints, or in-depth details of how this works at the Tortoise side.