search

matrix-org / rust-synapse-compress-state

A tool to compress some state in a Synapse instance's database
https://pypi.org/project/synapse-auto-compressor/
Apache License 2.0
143 stars 33 forks source link

Added option to only run the compressor on a range of state groups in a room #44

Closed Azrenbeth closed 3 years ago

Azrenbeth commented 3 years ago

BUILDS ON azren/added_graph_option (PR #43)

This allows the compressor to be called on only a selection of state groups in a room.

-b can be used to select the state group number to start compressing from (entering 0 or omitting this will start at the first state_group in the room)

-n can be used to enter (roughly) how many state groups you wish to compress.

The compressor will print to stdout where it got up to. Entering this state group number on the next run of the compressor will start from where the previous run finished

(NOTE: chunking produces worse compression than running all in one go, but will save memory)

erikjohnston commented 3 years ago

NOTE - changes meaning of -s option from last state group to compress to first state group

Why are we removing the option? I'd much rather we kept it and used a different name for the new option? That way we retain backwards compat.

Azrenbeth commented 3 years ago

NOTE - changes meaning of -s option from last state group to compress to first state group

Why are we removing the option? I'd much rather we kept it and used a different name for the new option? That way we retain backwards compat.

I removed it because I thought "max_state_group" was there just to limit the number of groups you work on at once, which the chunking options replace with something better, and removing it simplified the code a bit. And since there's no reason not to run the compressor on the state groups of the most recent messages (cos changing their deltas won't take THAT long) I couldn't think of a reason to keep it.

(So mainly I just forgot to think about backwards compatability...!)

Anyway, I'll add it back in!