Cleaning up Disk Space

[ArneNostitz]

Matrix Server:

• OS: [e.g. Ubuntu 21.04]

Problem description:

I only have 30GB of disk space and matrix is filling it up now nearly 99% ... there is a lot of media (20gb) that could be purged ... (dont know where the rest 10gb are hiding thou) ...

with the dockers ansible playbook i was able to run this matrix server the past few years and it was great ... but the help files regarding cleanup are too complicated to understand (the synapse api and stuff ... that is beyond my linux/terminal knowledge) ... is there a "Matrix cleanup for dummies" somewhere to be found? or a playbook tag like "delete-remote-media-olderThan100Days"?

sadly my matrix server is really unresponsive and nothing really works when the disk is full ... obviously ... and that usually breaks everything when i am heavily needing it :)

please a hint in the right direction ... best to a step by step tutorial ... thanks a lot!! chhers

[moan0s]

Sadly there is no real "cleanup for dummies" when it comes to media. Nevertheless I can maybe give some pointers:

• Here is a guide to maintain synapse as deployed by this playbook. Interesting for you is "Purging old data with the Purge History API" (not that easy to do) and "Compressing state" (easy) https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/maintenance-synapse.md#synapse-maintenance
• Reduce database size by vacuuming: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/maintenance-postgres.md#vacuuming-postgresql

• Mount an additional volume to the server that stores media. That is the solution I went for. It is quite easy if you use Hetzner or others that allow you to just add a volume. What you then do is

  • Shut down synapse
  • Add the following to you vars.yml

    
    #################################
    ## CUSTOM MEDIA STORE LOCATION ##
    #################################
    matrix_synapse_storage_path: "/mnt/yourvolume/matrix-storage"

  ###################################

  CUSTOM POSTGRES DATA LOCATION

  ################################### devture_postgres_data_path: "/mnt/yourvolume/postgres-data"

  + Move you files

  mv /matrix/synapse/data /mnt/yourvolume/matrix-storage mv /matrix/postgres/data /mnt/yourvolume/matrix-storage

  
  Pleas check these commands, I did not test these!

I hope this somehow helps, feel free to ask questions :)

[ArneNostitz]

i don't want to delete the history ... actually just deleting files that i uploaded would be sufficient at the beginning.

i am the only one on that server. i have a few bridges to Whatsapp and signal and others. but that's it.

is there an easy way to delete files i uploaded?

[ArneNostitz]

and then probably it's a safe bet to also delete all files bridged over ... so basically every image and video file. (i would keep the audio msgs for now if that is an option)

[ArneNostitz]

thanks @moan0s for linking the Infos ... is it safe to just delete files in the filesystem?

[aaronraimist]

@ArneNostitz if you want to delete remote media, which is media that originated on other servers, then you should use the API https://matrix-org.github.io/synapse/latest/admin_api/media_admin_api.html#purge-remote-media-api

If you are only deleting local media, it doesn't really matter whether you use the API or manually delete it from the file system but using the API is an easy way to do it. https://matrix-org.github.io/synapse/latest/admin_api/media_admin_api.html#delete-local-media-by-date-or-size

[ArneNostitz]

where would I put those post commands? this is where my dummies knowledge ends ... and reading through the doc is not making it any more accessible for me sadly :-)

[moan0s]

From here

Synapse's Admin API is not exposed to the internet by default. To expose it you will need to add matrix_nginx_proxy_proxy_matrix_client_api_forwarded_location_synapse_admin_api_enabled: true to your vars.ymlfile and rerun the playbook. So you need to set this and then your server will expose the admin API. An API ist something where you send commands to (via HTTP requests). So you can send a command from your local computer.

You first need to obtain an access token

And then use the following command

curl -X POST --header "Authorization: Bearer <access_token>" "https://your-matrix-server.com/_synapse/admin/v1/purge_media_cache?before_ts=1673592635000"

You need to replace the domain to your synapse address (should probably be matrix.yourdomain.org).

The command will purge all remote media from your server that is older than 1 month (from today, you would need to exchange the before_ts value. Useful conversion tool: https://www.unixtimestamp.com/ (make sure to multiply by 1000 to get milliseconds)

[ArneNostitz]

thank you so much! this helped a lot. 18 of 30 gb are free now. :) remote media purge did not delete anything thou ... only local. but i dont care as these things are either not important anymore or i can maybe get them back on whatsapp/signal/telegram

a easy way to do this in the playbook would be really cool, but with this explainer it was managable for me too! so again: thank you @moan0s

[aine-etke]

step-by-step guide: https://etke.cc/help/extras/synapse-admin/#remove-old-media