Delete previous messages based on their content, these messages were sent before matrix-commander establishment

[Benjamin-Loison]

Related to YouTube-operational-API/issues/237.

[Benjamin-Loison]

I just had a similar issue with OC3K notifyOnStackExchangeNotification.py.

[Benjamin-Loison]

On the already extablished LemnosLife VPS benjamin user matrix-commander -t 3 does not return anything.

matrix-commander --os-notify

returns silently.

matrix-commander -l

This program is ready and listening for its Matrix messages. To stop program type Control-C on keyboard or send signal to process 1778108. PID can also be found in file "/home/benjamin/.run/matrix-commander.a50fa1e2-2ad1-4b48-9cc3-f116a6972628.pid".

but there is no additional content when both ends send a message to each other...

matrix-commander -t 3 --print-event-id

start returning interesting results.

In fact it seems by default to only consider sent messages. Make the other end send a message make matrix-commander -t 3 work for received messages. Adding -y solves this issue.

However, only seems to retrieve the ciphertext and not the plaintext:

Message received for room CENSORED [!CENSORED:matrix.org] | sender CENSORED [@overclock3000:matrix.org] | 2024-04-11 15:55:25 | Received unknown event: MegolmEvent(source={'content': {'algorithm': 'm.megolm.v1.aes-sha2', 'ciphertext': 'CENSORED', 'device_id': 'CENSORED', 'sender_key': 'CENSORED', 'session_id': 'CENSORED'}, 'origin_server_ts': 1712843725080, 'room_id': '!CENSORED:matrix.org', 'sender': '@overclock3000:matrix.org', 'type': 'm.room.encrypted', 'unsigned': {'age': 183068}, 'event_id': 'CENSORED', 'user_id': '@overclock3000:matrix.org', 'age': 183068}, event_id='CENSORED', sender='@overclock3000:matrix.org', server_timestamp=1712843725080, decrypted=False, verified=False, sender_key='CENSORED', session_id='CENSORED', transaction_id=None, device_id='CENSORED', ciphertext='CENSORED', algorithm='m.megolm.v1.aes-sha2', room_id='!CENSORED:matrix.org')

matrix-commander -h

-l [NEVER|ONCE|FOREVER|TAIL|ALL], --listen [NEVER|ONCE|FOREVER|TAIL|ALL]
  Print received messages and listen to messages.
-t [NUMBER], --tail [NUMBER]
  Print last messages.
-y, --listen-self
  Print your own messages as well.
--print-event-id
  Print event ids of received messages. 

matrix-commander --listen tail

2024-04-11 16:08:34,685:    ERROR: matrix-commander: E240: An integer 1 or larger must be specified with --tail (0).
2024-04-11 16:08:34,686:     INFO: matrix-commander: 1 error and 0 warnings occurred.

matrix-commander --listen TAIL

2024-04-11 16:09:05,726:    ERROR: matrix-commander: E240: An integer 1 or larger must be specified with --tail (0).
2024-04-11 16:09:05,726:     INFO: matrix-commander: 1 error and 0 warnings occurred.

Seems related to matrix-commander/issues/47.

Just being able to see in plaintext new messages would be a good start. However, messages even sent by matrix-commander do not seem retrievable as plaintext.

Should investigate:

--sync FULL|OFF
  Choose synchronization options.

[Benjamin-Loison]

Giving a try to Element desktop Export Chat with Format JSON. Let us first make sure that given an id can delete the given message.

matrix-commander --room-redact '!CENSORED:matrix.org' 'CENSORED'

works.

list_messages_to_delete.py:

#!/usr/bin/python3

import json

with open('messages.json') as f:
    data = json.load(f)

messages = data['messages']
for message in messages:
    body = message['content'].get('body', '')
    if 'is not reachable by keyboard' in body:
        print(message['event_id'])

cat messages_to_delete.txt | while read line
do
    matrix-commander --room-redact '!CENSORED:matrix.org' "$line"
done

does the trick but face:

2024-04-11 17:29:41,804:    ERROR: matrix-commander: E213: Failed to redact event CENSORED in room !CENSORED:matrix.org with reason 'None'. Response: RoomRedactError: M_LIMIT_EXCEEDED Too Many Requests - retry after 354ms

Test body equality with a given message provided and verified by its event id to be more precise seems to make sense.

Should integrate directly matrix-commander --room-redact to the Python script and check its output to wait the given time.

matrix_commander_room_redact.py:

#!/usr/bin/python3

import json
import subprocess
import shlex
from tqdm import tqdm

# Assuming having joined a single room, which is my bot case.
ROOM_ID = subprocess.check_output('matrix-commander --joined-rooms', shell = True).decode('ascii')[:-1]
REASON = 'Spam'

with open('messages.json') as f:
    data = json.load(f)

messages = data['messages']
eventIds = [message['event_id'] for message in messages if 'selenium.common.exceptions.TimeoutException: Message: Navigation timed out after 300000 ms' in message['content'].get('body', '')]

for eventId in tqdm(eventIds):
    print(f'{eventId = }')
    output = subprocess.check_output(f'matrix-commander --room-redact {shlex.quote(ROOM_ID)} {shlex.quote(eventId)} {shlex.quote(REASON)}', shell = True)
    print(f'{output = }')

[Benjamin-Loison]

ROOM=`matrix-commander --joined-rooms`
matrix-commander --room-redact "$ROOM" '$CENSORED' '' "$ROOM" '$CENSORED' ''

[Benjamin-Loison]

matrix_commander_room_redact.py:

#!/usr/bin/python3

import json
import subprocess
import shlex

ROOM_ID = subprocess.check_output('matrix-commander --joined-rooms', shell = True).decode('ascii')[:-1]
REASON = 'Spam'

with open('messages.json') as f:
    data = json.load(f)

messages = data['messages']
eventIds = [message['event_id'] for message in messages if 'selenium.common.exceptions.TimeoutException: Message: Navigation timed out after 300000 ms' in message['content'].get('body', '')]

arguments = ' '.join([' '.join([shlex.quote(ROOM_ID), shlex.quote(eventId), shlex.quote(REASON)]) for eventId in eventIds])
command = f'matrix-commander --room-redact {arguments}'
print(f'{command = }')
output = subprocess.check_output(command, shell = True)
print(f'{output = }')

[Benjamin-Loison]

eventId = '$CENSORED'
2024-05-17 12:23:38,484:     INFO: matrix-commander: Successfully redacted event $CENSORED in room !CENSORED:matrix.org providing reason 'Spam'.
output = b''
 91%|█████████████████████████████████████████████████████████████████████████████████████████████         | 83/91 [01:45<00:26,  3.25s/it]eventId = '$CENSORED'
2024-05-17 12:23:41,438:    ERROR: matrix-commander: E213: Failed to redact event $CENSORED in room !CENSORED:matrix.org with reason 'Spam'. Response: RoomRedactError: M_LIMIT_EXCEEDED Too Many Requests - retry after 87ms
2024-05-17 12:23:41,440:     INFO: matrix-commander: 1 error and 0 warnings occurred.
 91%|█████████████████████████████████████████████████████████████████████████████████████████████         | 83/91 [01:48<00:10,  1.30s/it]
Traceback (most recent call last):
  File "/home/benjamin/./a.py", line 21, in <module>
    output = subprocess.check_output(f'matrix-commander --room-redact {shlex.quote(ROOM_ID)} {shlex.quote(eventId)} {shlex.quote(REASON)}', shell = True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'matrix-commander --room-redact '!CENSORED:matrix.org' '$CENSORED' Spam' returned non-zero exit status 1.