search

element-hq / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://element-hq.github.io/synapse
GNU Affero General Public License v3.0
1.58k stars 198 forks source link

Failure when downloading any remote media #17927

Open Kimiblock opened 6 days ago

Kimiblock commented 6 days ago

Description

Synapse fails to download any remote media

Steps to reproduce

Homeserver

moechat.kimiblock.top

Synapse Version

1.117

Installation Method

Other (please mention below)

Database

PostgreSQL single server

Workers

Single process

Platform

Arch Linux rolling, all packages has been updated

Configuration

experimental_features: msc3575_enabled: true msc3852_enabled: true msc3026_enabled: true msc3266_enabled: true msc2815_enabled: true msc2815_enabled: true msc3773_enabled: true msc3664_enabled: true msc3881_enabled: true

Push

msc3890_enabled: true msc3881_enabled: true msc3391_enabled: true msc4028_push_encrypted_events: true

enable_media_repo: true

Keep this off!

enable_authenticated_media: false

media_store_path: "media_repo" max_pending_media_uploads: 4 max_upload_size: 3M max_image_pixels: 32M dynamic_thumbnails: true media_retention: local_media_lifetime: 14d remote_media_lifetime: 4d

web_client_location: https://plane.kimiblock.top

presence: enabled: true

Relevant log output

2024-11-13 08:44:51,926 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:51,927 - synapse.media.media_r
epository - 910 - ERROR - GET-5074 - Failed to 
fetch remote media 2o4.ca/d14KyhF8hOafktomhOlRt
wC2zK0SiZlM
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
2024-11-13 08:44:52,763 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:52,763 - synapse.media.media_r
epository - 910 - ERROR - GET-5044 - Failed to 
fetch remote media uwu.zemos.net/6Oc7VA1lboOpV6
qltdMeoZ21W9BR79Rv
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
2024-11-13 08:44:57,462 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:57,462 - synapse.media.media_r
epository - 910 - ERROR - GET-5153 - Failed to 
fetch remote media nichi.co/BJghvnTXFaPPPPuVomq
GOyCo
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
2024-11-13 08:44:58,430 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:58,430 - synapse.media.media_r
epository - 910 - ERROR - GET-5160 - Failed to 
fetch remote media nichi.co/BJghvnTXFaPPPPuVomq
GOyCo
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
➜  ~

Anything else that would be useful to know?

No response

erikjohnston commented 4 days ago

That is really odd, the error is:

AttributeError: 'MultipartParser' object has no attribute 'write'

Which is a class from python_multipart package. It's possible that another package is colliding with it, which doesn't have write? You should try re-installing python-multipart perhaps?

Using a system-wide installation can be slightly brittle, as multiple python packages can clash, i.e. two different packages can install modules under multipart.

Kimiblock commented 4 days ago

I have both multipart and python-multipart installed, that is the cause.

alexanderustinov commented 4 days ago

I've had the same with v1.119.0 on gentoo with distro packages. It seems, synapse uses both multipart and python-multipart (doesn't start with multipart removed) Workarounded problem by changing https://github.com/element-hq/synapse/blob/850ff14613040d733ab28373a54c95e20182cd1c/synapse/http/client.py#L39 to

import python_multipart as multipart
Kimiblock commented 4 days ago

https://github.com/element-hq/synapse/pull/17932

arkamar commented 4 days ago

@alexanderustinov that sounds weird, because gentoo package patches the line as you suggest, see https://github.com/gentoo/gentoo/blob/67abb5404860cf0bdb60b2454e1b98a3b5fb881c/net-im/synapse/synapse-1.119.0.ebuild#L181-L183 and all tests pass for ~amd64. We do this change since v1.116.0-r2.

alexanderustinov commented 3 days ago

@alexanderustinov that sounds weird, because gentoo package patches the line as you suggest, see https://github.com/gentoo/gentoo/blob/67abb5404860cf0bdb60b2454e1b98a3b5fb881c/net-im/synapse/synapse-1.119.0.ebuild#L181-L183 and all tests pass for ~amd64. We do this change since v1.116.0-r2.

@arkamar i've had custom ebuild in an overlay for years. Haven't noticed synapse in ::gentoo, thanks.