search

mautrix / telegram

A Matrix-Telegram hybrid puppeting/relaybot bridge
https://matrix.to/#/#telegram:maunium.net
GNU Affero General Public License v3.0
1.3k stars 172 forks source link

Parallel transfer breaks if Matrix media repo doesn't include Content-Length header #411

Open bttmchnd opened 4 years ago

bttmchnd commented 4 years ago 
tamara_1        | [2020-01-27 03:10:46,661] [DEBUG@mau.init] Startup actions complete in 0.77 seconds, now running forever
tamara_1        | [2020-01-27 03:11:16,289] [DEBUG@mau.mx] Received event: MessageEvent(type=EventType("m.sticker", EventType.Class.MESSAGE), room_id='!mQDlWhEQkHYVqWaxdV:sunbutt.faith', event_id='$p64XaimoTam_Yghj9qk77GZSG4b5z20JaYQosak7W8A', sender='@commagray:sunbutt.faith', timestamp=1580094676215, content=MediaMessageEventContent(msgtype=m.sticker, body='😂', external_url=None, _relates_to=None, url='mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', info={'dimension': {'author': {'name': 'Telegram', 'reference': 'https://t.me/addstickers/DevStickersPack01', 'type': 'telegram'}, 'license': {'name': 'Telegram', 'urlPath': '/licenses/telegram-imported.txt'}}, 'h': 256, 'mimetype': 'image/png', 'thumbnail_info': {'h': 256, 'mimetype': 'image/png', 'w': 256}, 'thumbnail_url': 'mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', 'w': 256, '__mautrix_msgtype': 'm.sticker'}), unsigned=MessageUnsigned(age=63, transaction_id=None))
tamara_1        | [2020-01-27 03:11:16,300] [DEBUG@mau.mx] Received Matrix event "MediaMessageEventContent(msgtype=m.sticker, body='😂', external_url=None, _relates_to=None, url='mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', info={'dimension': {'author': {'name': 'Telegram', 'reference': 'https://t.me/addstickers/DevStickersPack01', 'type': 'telegram'}, 'license': {'name': 'Telegram', 'urlPath': '/licenses/telegram-imported.txt'}}, 'h': 256, 'mimetype': 'image/png', 'thumbnail_info': {'h': 256, 'mimetype': 'image/png', 'w': 256}, 'thumbnail_url': 'mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', 'w': 256, '__mautrix_msgtype': 'm.sticker'})" from @commagray:sunbutt.faith in !mQDlWhEQkHYVqWaxdV:sunbutt.faith
tamara_1        | [2020-01-27 03:11:16,313] [INFO@aiohttp.access] 192.168.240.3 [27/Jan/2020:03:11:16 +0000] "PUT /transactions/3170?access_token=REDACTED HTTP/1.1" 200 158 "-" "Synapse/1.9.0"
tamara_1        | [2020-01-27 03:11:16,336] [ERROR@mau.as] Exception in Matrix event handler
tamara_1        | Traceback (most recent call last):
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix/appservice/appservice.py", line 229, in try_handle
tamara_1        |     await handler_func(event)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 319, in int_handle_event
tamara_1        |     await self.handle_message(evt.room_id, evt.sender, evt.content, evt.event_id)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 237, in handle_message
tamara_1        |     await portal.handle_matrix_message(sender, message, event_id)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/portal/matrix.py", line 389, in handle_matrix_message
tamara_1        |     await self._handle_matrix_file(sender_id, event_id, space, client, content, reply_to,
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/portal/matrix.py", line 254, in _handle_matrix_file
tamara_1        |     file_handle, file_size = await parallel_transfer_to_telegram(client, self.main_intent,
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 298, in parallel_transfer_to_telegram
tamara_1        |     return await _internal_transfer_to_telegram(client, response)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 266, in _internal_transfer_to_telegram
tamara_1        |     part_size, part_count, is_large = await uploader.init_upload(file_id, file_size)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 199, in init_upload
tamara_1        |     connection_count = connection_count or self._get_connection_count(file_size)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 135, in _get_connection_count
tamara_1        |     if file_size > full_size:
tamara_1        | TypeError: '>' not supported between instances of 'NoneType' and 'int'

But they work from Telegram to Matrix.

Version: latest Docker image.

a22sc commented 3 years ago

Getting a similar behavior with a pdf file

[2020-10-13 12:00:36,051] [ERROR@mau.as] Exception in Matrix event handler 
Traceback (most recent call last): 
  File "/opt/mautrix-telegram/lib/python3.8/site-packages/mautrix/appservice/as_handler.py", line 157, in try_handle 
    await handler_func(event) 
  File "/opt/mautrix-telegram/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 417, in int_handle_event 
    await self.handle_message(evt.room_id, evt.sender, evt.content, evt.event_id) 
  File "/opt/mautrix-telegram/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 278, in handle_message 
    await portal.handle_matrix_message(sender, message, event_id) 
  File "/opt/mautrix-telegram/mautrix_telegram/portal/matrix.py", line 369, in handle_matrix_message 
    await self._handle_matrix_message(sender, content, event_id) 
  File "/opt/mautrix-telegram/mautrix_telegram/portal/matrix.py", line 416, in _handle_matrix_message 
    await self._handle_matrix_file(sender_id, event_id, space, client, content, reply_to, 
  File "/opt/mautrix-telegram/mautrix_telegram/portal/matrix.py", line 255, in _handle_matrix_file 
    w, h = content.info.width, content.info.height 
AttributeError: 'FileInfo' object has no attribute 'width'

on latest master branch

tulir commented 3 years ago

@a000a that's a completely unrelated problem, fixed in fdc58ce450f3768bc9cbef4d62fde130dde8ef05

tulir commented 3 years ago

So the problem here is that the parallel transfer option only works if the media repo includes a Content-Length header. I think matrix-media-repo doesn't include it in some cases, Synapse probably always does. A temporary fix is to disable parallel transfers in the config. The proper solution is to automatically fall back to normal file transfer when there's no content length header.

Ugzuzg commented 2 years ago

Seems to be breaking stickers (Matrix -> Telegram), too: image

First one was sent with parallel_file_transfer: true, second with parallel_file_transfer: false.

tulir commented 2 years ago

That's expected, the bridge can't convert images to webp without having the whole thing in memory. If the stickers are in the correct format already on Matrix, then they should work on Telegram too.

Ugzuzg commented 2 years ago

All the default dimension stickers use PNG, and it even converts the stickers imported from Telegram to PNG. Will you accept a PR that ignores parallel_file_transfer setting for stickers?