search

matrix-org / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://matrix-org.github.io/synapse
Apache License 2.0
11.83k stars 2.13k forks source link

GET /room_keys/keys/{roomId}/{sessionId} fails if the session ID has a "/" slash #16523

Closed krille-chan closed 1 year ago

krille-chan commented 1 year ago

Description

A call at GET /_matrix/client/v3/room_keys/keys/{roomId}/{sessionId} returns a "404" if the session ID has a / slash character in it which needs to be URI escaped.

Example of such a session ID: lDUtT%2FGIitTHgCyetqI%2FtWn8odn5crfafJ3jOaUb6Rw

For sessions which do not have such a slash in it the endpoint works fine. This leads to the problem that randomly sessions cannot be fetched.

Steps to reproduce

Homeserver

janian.de

Synapse Version

1.94.0

Installation Method

pip (from PyPI)

Database

PostgreSQL

Workers

Single process

Platform

Ubuntu 20.04

Configuration

No response

Relevant log output

Session not found

Anything else that would be useful to know?

No response

clokep commented 1 year ago

Can you please provide the Synapse logs for when this occurs?

What is the reverse proxy configuration of the homeserver? I'm wondering if it is decoding or normalizing URLs (see the note at https://matrix-org.github.io/synapse/latest/reverse_proxy.html).

krille-chan commented 1 year ago

Thanks for the quick answer. I will try to get the logs. Will take a little bit. The reverse proxy config of apache2 is:

<VirtualHost *:80>
serverName janian.de
serverAlias www.janian.de
ServerAdmin webmaster@localhost
DocumentRoot /var/www/nextcloud
Alias /nextcloud "/var/www/nextcloud/"
ProxyPreserveHost On
SSLProxyEngine On
SSLProxyVerify none
SSLProxyCheckPeerCN off
AllowEncodedSlashes NoDecode

<IfModule mod_headers.c>
Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
</IfModule>

<Directory "/var/www/nextcloud/">
Options +FollowSymlinks
AllowOverride All

<IfModule mod_dav.c>
Dav off
</IfModule>

Require all granted

SetEnv HOME /var/www/html
SetEnv HTTP_HOME /var/www/html

</Directory>

<Location /.well-known/matrix/client>
    ErrorDocument 200 '{"m.homeserver":{"base_url":"https://janian.de"}}'
    Redirect 200 /
    Header always set Content-Type application/json
    Header always set Access-Control-Allow-Origin *
</Location>

...

<Location "/_matrix/">
  ProxyPass "http://localhost:8008/_matrix/"
  SetEnv force-proxy-request-1.0 1
  SetEnv proxy-nokeepalive 1
</Location>

...
krille-chan commented 1 year ago

Here are the logs, where the client requests 6 keys and can only load 2 (those without at slash in it)

sec/0.000sec/0) 59B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/lDUtT/GIitTHgCyetqI/tWn8odn5crfafJ3jOaUb6Rw?version=252 HTTP/1.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" [0 dbevts]
2023-10-19 16:21:51,655 - synapse.http.server - 124 - INFO - GET-28091 - <XForwardedForRequest at 0x7fbfbc10ba60 method='GET' uri='/_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/lDUtT%252FGIitTHgCyetqI%252FtWn8odn5crfafJ3jOaUb6Rw?version=252' clientproto='HTTP/1.0' site='8008'> SynapseError: 404 - No room_keys found
2023-10-19 16:21:51,656 - synapse.access.http.8008 - 465 - INFO - GET-28091 - 2a02:3032:2e0:9b55:a174:6241:d3d:4c82 - 8008 - {@cat:janian.de} Processed request: 0.003sec/0.000sec (0.002sec, 0.000sec) (0.000sec/0.001sec/2) 54B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/lDUtT%252FGIitTHgCyetqI%252FtWn8odn5crfafJ3jOaUb6Rw?version=252 HTTP/1.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" [0 dbevts]
2023-10-19 17:00:21,405 - synapse.http.server - 124 - INFO - GET-30258 - <XForwardedForRequest at 0x7fbfcc306a00 method='GET' uri='/_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/YRtSs6eGvXfd3QQK33RwhTg8vyS%252F0cDjhraaZ2JgBUM?version=252' clientproto='HTTP/1.0' site='8008'> SynapseError: 404 - No room_keys found
2023-10-19 17:00:21,406 - synapse.access.http.8008 - 465 - INFO - GET-30258 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.005sec/0.001sec (0.002sec, 0.000sec) (0.000sec/0.002sec/2) 54B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/YRtSs6eGvXfd3QQK33RwhTg8vyS%252F0cDjhraaZ2JgBUM?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
2023-10-19 17:00:21,407 - synapse.http.server - 124 - INFO - GET-30259 - <XForwardedForRequest at 0x7fbfbc5d1fd0 method='GET' uri='/_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/L%252FTSr5twVz+GE22dPYqtcBOtN7HCFi+C0zlZvM1b%252FQ4?version=252' clientproto='HTTP/1.0' site='8008'> SynapseError: 404 - No room_keys found
2023-10-19 17:00:21,408 - synapse.access.http.8008 - 465 - INFO - GET-30259 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.007sec/0.001sec (0.001sec, 0.000sec) (0.000sec/0.001sec/2) 54B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/L%252FTSr5twVz+GE22dPYqtcBOtN7HCFi+C0zlZvM1b%252FQ4?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
2023-10-19 17:00:21,410 - synapse.http.server - 124 - INFO - GET-30260 - <XForwardedForRequest at 0x7fbf7845a9a0 method='GET' uri='/_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/8xL257hNr7rBCfxUroH6%252F5U4Dm+cl1lbDy8OTOiZ9GA?version=252' clientproto='HTTP/1.0' site='8008'> SynapseError: 404 - No room_keys found
2023-10-19 17:00:21,411 - synapse.access.http.8008 - 465 - INFO - GET-30260 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.008sec/0.001sec (0.001sec, 0.000sec) (0.000sec/0.001sec/2) 54B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/8xL257hNr7rBCfxUroH6%252F5U4Dm+cl1lbDy8OTOiZ9GA?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
2023-10-19 17:00:21,412 - synapse.http.server - 124 - INFO - GET-30261 - <XForwardedForRequest at 0x7fbf98175700 method='GET' uri='/_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/lDUtT%252FGIitTHgCyetqI%252FtWn8odn5crfafJ3jOaUb6Rw?version=252' clientproto='HTTP/1.0' site='8008'> SynapseError: 404 - No room_keys found
2023-10-19 17:00:21,413 - synapse.access.http.8008 - 465 - INFO - GET-30261 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.010sec/0.000sec (0.001sec, 0.000sec) (0.000sec/0.001sec/2) 54B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/lDUtT%252FGIitTHgCyetqI%252FtWn8odn5crfafJ3jOaUb6Rw?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
2023-10-19 17:00:21,419 - synapse.http.server - 124 - INFO - GET-30262 - <XForwardedForRequest at 0x7fbf78736970 method='GET' uri='/_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/CdrdHx52Uv7zuI95QCEu7WrrcUp1H1aoiULCwpXi%252F8I?version=252' clientproto='HTTP/1.0' site='8008'> SynapseError: 404 - No room_keys found
2023-10-19 17:00:21,419 - synapse.access.http.8008 - 465 - INFO - GET-30262 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.004sec/0.001sec (0.001sec, 0.000sec) (0.000sec/0.002sec/2) 54B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/CdrdHx52Uv7zuI95QCEu7WrrcUp1H1aoiULCwpXi%252F8I?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
2023-10-19 17:00:21,422 - synapse.access.http.8008 - 465 - INFO - GET-30263 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.005sec/0.001sec (0.001sec, 0.000sec) (0.000sec/0.001sec/2) 795B 200 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/7dxxuhVnS8qWyhxPX6+GH6TE1oHEGGn6eKyDBUpM6ks?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
2023-10-19 17:00:21,424 - synapse.access.http.8008 - 465 - INFO - GET-30264 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.007sec/0.001sec (0.000sec, 0.000sec) (0.000sec/0.001sec/2) 795B 200 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/ggpJ8CMTegQ+GJOws1l7Nu7gLrwhtZszp9rStQ0Smkg?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
2023-10-19 17:00:21,426 - synapse.http.server - 124 - INFO - GET-30265 - <XForwardedForRequest at 0x7fbf584e7460 method='GET' uri='/_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/1AB2OMu5xO29nPQfR5wQUF7iRRH0Z8VP87%252FYtCXqIg0?version=252' clientproto='HTTP/1.0' site='8008'> SynapseError: 404 - No room_keys found
2023-10-19 17:00:21,426 - synapse.access.http.8008 - 465 - INFO - GET-30265 - 46.114.180.75 - 8008 - {@cat:janian.de} Processed request: 0.009sec/0.000sec (0.001sec, 0.000sec) (0.000sec/0.001sec/2) 54B 404 "GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/1AB2OMu5xO29nPQfR5wQUF7iRRH0Z8VP87%252FYtCXqIg0?version=252 HTTP/1.0" "Dart/3.1 (dart:io)" [0 dbevts]
clokep commented 1 year ago 
GET /_matrix/client/v3/room_keys/keys/!QMgjfyOecwAwxBzPYW:janian.de/lDUtT/GIitTHgCyetqI/tWn8odn5crfafJ3jOaUb6Rw?version=252

This looks like either:

  1. Synapse is getting the decoded request (as I mentioned above).
  2. The client is not properly URL encoding the parameter. (What client are you seeing this with?)

It could be worth checking e.g. /sync requests, locally I see:

2023-10-19 11:19:16,037 - synapse.access.http.8080 - 465 - INFO - PUT-192 - 127.0.0.1 - 8080 - {@alice:localhost:8480} Processed request: 0.001sec/0.000sec (0.000sec, 0.000sec) (0.000sec/0.000sec/0) 2B 200 "PUT /_matrix/client/v3/presence/%40alice%3Alocalhost%3A8480/status HTTP/1.1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0" [0 dbevts]

Note %40alice%3Alocalhost%3A8480 and not @alice:localhost:8480.

krille-chan commented 1 year ago

Hey thanks for the response. I have looked into the docs and tried to change the line

ProxyPass "http://localhost:8008/_matrix/"

to:

ProxyPass "http://localhost:8008/_matrix/" nocab

and this seems to fix the issue. So looks like it was my own RTFM problem... thank you so much for the info and sorry for the time