search

matrix-org / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://matrix-org.github.io/synapse
Apache License 2.0
11.83k stars 2.12k forks source link

Request & follow redirects for /media/v3/download #16701

Closed clokep closed 11 months ago

clokep commented 11 months ago

This has two mostly unrelated changes to it:

I tested this by setting up a server locally and doing:

curl --header "Authorization: Bearer $TOK" http://localhost:8080/_matrix/media/v3/download/beeper.com/18850ea089e0ecc16d7db55527925b43ad63295c

Which had the following logs:

synapse.http.matrixfederationclient - 665 - DEBUG - GET-0 - {GET-O-1} [beeper.com] Sending request: GET matrix-federation://beeper.com/_matrix/media/v3/download/beeper.com/18850ea089e0ecc16d7db55527925b43ad63295c?allow_remote=false&timeout_ms=20000&allow_redirect=true; timeout 60.000000s
synapse.http.matrixfederationclient - 665 - DEBUG - GET-0 - {GET-O-2} [beeper.com] Sending request: GET https://thumbnails.eu-central-2.media.beeper.com/8ca/7d9a66bed41b67440ab3807a03e70e22ac70b; timeout 60.000000s
synapse.http.matrixfederationclient - 716 - DEBUG - GET-0 - {GET-O-2} [beeper.com] Got response headers: 200 OK
synapse.http.matrixfederationclient - 1491 - INFO - GET-0 - {GET-O-1} [beeper.com] Completed: 200 OK [140970 bytes] GET matrix-federation://beeper.com/_matrix/media/v3/download/beeper.com/18850ea089e0ecc16d7db55527925b43ad63295c?allow_remote=false&timeout_ms=20000&allow_redirect=true

You can see it attempts to fetch the media from Beeper's homeserver, gets a redirect to a CDN and then fetches it from there instead.

In order to accomplish this I reworked the fetching of downloaded media to be more similar to other federation requests. You can compare the solid to the dotted line. The main goal of this was to add the fallback from v3 -> r0.

flowchart
    A[MediaRepository._download_remote_file]
    B[FederationClient.download_media]
    C1[TransportLayerClient.download_media_r0]
    C2[TransportLayerClient.download_media_v3]
    D[MatrixFederationHttpClient.get_file]
    A --> D

    A -.-> B -.-> C2 -.-> D
    B -.-> C1 -.-> D

Fixes #15196, part of #15661.

clokep commented 11 months ago

Something like https://github.com/matrix-org/synapse-s3-storage-provider/issues/107 might be useful if you're interested in supporting this for the Client-Server API. Implementing it in Synapse proper seems out of scope?