search

matrix-org / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://matrix-org.github.io/synapse
Apache License 2.0
11.78k stars 2.13k forks source link

Quotas for file and event storage #3339

Open ara4n opened 6 years ago

ara4n commented 6 years ago

Something that keeps coming up is the need to track which users are responsible for chewing all the storage on your server - whether that's by storing lots of stuff in the media repo, or by joining massive rooms and chewing all your DB (or submitting loads of events).

We probably need to distinguish storage which the user created directly (i.e. by sending a file or event) or indirectly (by accessing a file or a room).

We probably care more about directly attributable storage costs, as indirect ones are going to be shared across a bunch of users - plus tracking the indirect ones could result in a metadata leak for e2e rooms (i.e. which users have access to the files in a given room)

stonie08 commented 5 years ago

Having some kind of per-user storage-quota would be a really nice addition. I also see this as a possible issue for larger hosted instances as having unrestricted media storage might result in the service being misused as unlimited cloud storage. Even in self-hosted-home-user-home-server settings it'd allow people to more easily hand-out accounts on their servers, without being worried about users taking up too much/all of their storage. It's much easier to do this if it's possible to allocate a maximum storage they can use.

Without deeper knowledge of the protocol/server or client implementations I'd personally like to see a configurable per-user/per-group storage quota, combined with some kind of client-side export of messages and media. Ideally it is easily possible for the user to backup all the old messages (e.g. all messages before dd/mm/yyyy) in some standardized format to make room for new messages/media. Maybe also a configuration option to decide on what happens if the quota is exceeded: Either receiving messages stops working or the oldest elements are removed. The seconds option might make for a smoother experience for users not interested in really old messages.

abeluck commented 4 years ago

We're also worried about our synapse instances being used as illicit file sharing platforms.

Our instances have a quite high max file upload (in the GBs). We're building on the synapse+matrix ecosystem to provide non-standard user experiences (not just whatsapp or slack like) and file sharing is an important part of that, but we're worried we can effectively be DoSed by bad actors looking for a file sharing avenue.

This isn't a hypothetical worry either, we ran open XMPP servers for years and this is very very common in that space.

clokep commented 4 years ago

We're also worried about our synapse instances being used as illicit file sharing platforms.

There are some tools in the admin APIs to quarantine media. Regardless it sounds like your issue isn't really about quotas but about managing content for other reasons. If so, could you please open a new issue with more details!