mautrix-telegram bridge getting rate limited

[chuckwagoncomputing]

Playbook Configuration:

The relevant portion of my vars.yml file looks like this:

matrix_mautrix_telegram_configuration_extension_yaml:
      bridge:
        backfill:
          enable: true
          double_puppet_backfill: true

Problem description:

When mautrix-telegram tries to backfill a chat with a long history, it gets rate limited. Here's the error message:

Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]: [2023-06-25 12:29:22,806] [ERROR@mau.portal.961975238<->493693808] Error in initial backfill
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]: Traceback (most recent call last):
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix_telegram/portal.py", line 1074, in _create_matrix_room
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     await self.forward_backfill(user, initial=True, client=client)
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix_telegram/portal.py", line 2866, in forward_backfill
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     output = await self.backfill(
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix_telegram/portal.py", line 2884, in backfill
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     return await self._locked_backfill(
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix_telegram/portal.py", line 2939, in _locked_backfill
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     insertion_id, event_count, message_count, lowest_id = await self._backfill_messages(
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix_telegram/portal.py", line 3186, in _backfill_messages
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     event_ids = [
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix_telegram/portal.py", line 3187, in <listcomp>
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     await intent.send_message_event(
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix/appservice/api/intent.py", line 142, in wrapper
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     return await __method(*args, **kwargs)
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix/appservice/api/intent.py", line 428, in send_message_event
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     return await super().send_message_event(room_id, event_type, content, **kwargs)
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix/client/api/events.py", line 450, in send_message_event
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     resp = await self.api.request(
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix/api.py", line 395, in request
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     resp_data, resp = await self._send(
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:   File "/usr/lib/python3.10/site-packages/mautrix/api.py", line 250, in _send
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]:     raise make_request_error(
Jun 25 07:29:22 fergus matrix-mautrix-telegram[75487]: mautrix.errors.request.MLimitExceeded: Too Many Requests

I don't understand why this happens. rate_limited:false is in the config https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/4dda00dbc609b0b16b3d2090fc43eb314733eecb/roles/custom/matrix-bridge-mautrix-telegram/defaults/main.yml#L155 My understanding is that this is supposed to disable rate limiting for all users of this appservice, which should include the @telegram_* users which mirror messages. Relevant Synapse code: https://github.com/matrix-org/synapse/blob/d53547352063c3e66302f7bdc40d7ac7270e605c/synapse/api/ratelimiting.py#L139

Additional context Backfilling with MSC2716 wasn't working at all for me, so this is using the timestamp massaging mode of mautrix-telegram. The resulting history is scrambled, which I assume is a result of it being rate limited, as it works perfectly on chats with less history.

[chuckwagoncomputing]

It just occurred to me that it's probably limiting my account - because of the double-puppeting, the messages I sent are bridged using my account.

[chuckwagoncomputing]

Could ratelimits be disabled in the register-user task?

[spantaleev]

Could ratelimits be disabled in the register-user task?

They cannot. You'd need to use one of the other ways

[chuckwagoncomputing]

So it can only be fixed manually, and not by default?