spantaleev / matrix-docker-ansible-deploy

🐳 Matrix (An open network for secure, decentralized communication) server setup using Ansible and Docker
GNU Affero General Public License v3.0
4.78k stars 1.03k forks source link

matrix-synapse-auto-compressor.service: Failed #3397

Closed felixx9 closed 1 month ago

felixx9 commented 3 months ago

Describe the bug A long time, "my" auto-compressor was running. Today I checked my server and I found this error:

$ systemctl status matrix-synapse-auto-compressor

● matrix-synapse-auto-compressor.service - Synapse State Auto Compressor
     Loaded: loaded (/etc/systemd/system/matrix-synapse-auto-compressor.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2024-07-06 00:00:08 UTC; 20h ago
TriggeredBy: ● matrix-synapse-auto-compressor.timer
    Process: 373527 ExecStartPre=/usr/bin/env sh -c /usr/bin/env docker stop --time=3 matrix-synapse-auto-compressor 2>/dev/null || true (code=exited, status=0/SUCCESS)
    Process: 373556 ExecStartPre=/usr/bin/env sh -c /usr/bin/env docker rm matrix-synapse-auto-compressor 2>/dev/null || true (code=exited, status=0/SUCCESS)
    Process: 373576 ExecStartPre=/usr/bin/env docker create --rm --name=matrix-synapse-auto-compressor --log-driver=none --cap-drop=ALL --read-only --user=998:1001 --network=matrix-postgres --env-file=/matrix/synapse-auto-compressor/env --entrypoint=/bin/sh registry.gitlab.com/etke.cc/rust-synapse-compress-state:v0.>
    Process: 373591 ExecStart=/usr/bin/env docker start --attach matrix-synapse-auto-compressor (code=exited, status=101)
   Main PID: 373591 (code=exited, status=101)
        CPU: 162ms

Jul 06 00:00:00 v2202210159252202628 systemd[1]: Starting Synapse State Auto Compressor...
Jul 06 00:00:00 v2202210159252202628 matrix-synapse-auto-compressor[373576]: 8b496798f69fcf07da9dc39242575f73ab0f909a35c13380b7261b9dc612f14d
Jul 06 00:00:01 v2202210159252202628 matrix-synapse-auto-compressor[373591]: [2024-07-06T00:00:01Z INFO  synapse_auto_compressor] synapse_auto_compressor started
Jul 06 00:00:01 v2202210159252202628 matrix-synapse-auto-compressor[373591]: [2024-07-06T00:00:01Z INFO  synapse_auto_compressor::manager] Running compressor on room !UPsqJAKJwKaqqYkxNZ:cutefunny.art with chunk size 500
Jul 06 00:00:07 v2202210159252202628 matrix-synapse-auto-compressor[373591]: [2024-07-06T00:00:07Z ERROR panic] thread 'main' panicked at 'Missing 1067109': src/lib.rs:668
Jul 06 00:00:08 v2202210159252202628 systemd[1]: matrix-synapse-auto-compressor.service: Main process exited, code=exited, status=101/n/a
Jul 06 00:00:08 v2202210159252202628 systemd[1]: matrix-synapse-auto-compressor.service: Failed with result 'exit-code'.
Jul 06 00:00:08 v2202210159252202628 systemd[1]: Failed to start Synapse State Auto Compressor.

not sure, whther this is a "this-playbook-issue"...

To Reproduce

spantaleev commented 3 months ago

Sounds like an issue with the tool itself. Consider reporting to its repository.

felixx9 commented 2 months ago

yeah thx. found an issue on that. Will try the almost "crazy" workaround: https://github.com/matrix-org/rust-synapse-compress-state/issues/78#issuecomment-1409932869

edit: the mentioned sql script fixes the issue for me.

Zocker1999NET commented 2 months ago

yeah thx. found an issue on that. Will try the almost "crazy" workaround: matrix-org/rust-synapse-compress-state#78 (comment)

That script also worked for me, but I adapted it slightly so that it "just works" on this deployment. But I also needed to adapt the postgres/bin/cli-non-interactive script because it did not worked on my setup (but I suspect that’s probably a "me" issue). See the patches attached.

Patches ```patch diff --git a/compress-fix.sh.old b/compress-fix.sh --- a/compress-fix.sh.old +++ b/compress-fix.sh @@ -15,16 +15,11 @@ set -e -u -o pipefail -export PGHOST=$1 -export PGDATABASE=$2 -export PGUSER=$3 -export PGPASSWORD=$4 - SCRIPT=$(basename $0) echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ') INFO ${SCRIPT}]" Starting -psql <<_EOF +/matrix/postgres/bin/cli-non-interactive synapse <<_EOF BEGIN; DELETE ``` ```patch diff --git a/matrix/postgres/bin/cli-non-interactive.old b/matrix/postgres/bin/cli-non-interactive --- a/matrix/postgres/bin/cli-non-interactive.old +++ b/matrix/postgres/bin/cli-non-interactive @@ -1,6 +1,7 @@ #!/bin/bash docker run \ + --interactive \ --rm \ --user=987:1002 \ --cap-drop=ALL \ ```
aine-etke commented 1 month ago

fixed: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3473