hibobmaster / matrix-stt-bot

A simple matrix bot that transcribes your voice to text message
https://matrix.to/#/#public:matrix.qqs.tw
MIT License
21 stars 3 forks source link

Error validating event: 'file' is a required property #5

Closed rhetenor closed 10 months ago

rhetenor commented 10 months ago

Hey,

Thanks so much for your work. I really appreciate this!

I've just setup the bot and encryption seems to be working correctly. But there is a problem with the message events. The schema requires the file attribute of the event which is according to the matrix spec required for encrypted messages. But somehow even though the message should be encrypted it is not there. I don't know if it is a problem with the bot or my matrix server instance but it happens with whatever audio message I send to the bot.

This is the output of the bot:

Error validating event: 'file' is a required property

Failed validating 'required' in schema['properties']['content']:
    {'properties': {'body': {'type': 'string'},
                    'file': {'properties': {'hashes': {'properties': {'sha256': {'type': 'string'}},
                                                       'type': 'object'},
                                            'iv': {'type': 'string'},
                                            'key': {'properties': {'alg': {'type': 'string'},
                                                                   'k': {'type': 'string'}},
                                                    'required': ['alg',
                                                                 'k'],
                                                    'type': 'object'},
                                            'url': {'type': 'string'}},
                             'required': ['url', 'hashes', 'iv', 'key'],
                             'type': 'object'},
                    'info': {'properties': {'thumbnail_file': {'properties': {'hashes': {'properties': {'sha256': {'type': 'string'}},
                                                                                         'type': 'object'},
                                                                              'iv': {'type': 'string'},
                                                                              'key': {'properties': {'alg': {'type': 'string'},
                                                                                                     'k': {'type': 'string'}},
                                                                                      'required': ['alg',
                                                                                                   'k'],
                                                                                      'type': 'object'},
                                                                              'url': {'type': 'string'}},
                                                               'required': ['url',
                                                                            'hashes',
                                                                            'iv',
                                                                            'key'],
                                                               'type': 'object'}},
                             'type': 'object'},
                    'msgtype': {'enum': ['m.image',
                                         'm.audio',
                                         'm.video',
                                         'm.file'],
                                'type': 'string'}},
     'required': ['body', 'file', 'msgtype'],
     'type': 'object'}

On instance['content']:
    {'body': 'audio.ogg',
     'info': {'duration': 68000,
              'mimetype': 'audio/ogg; codecs=opus',
              'size': 152320},
     'msgtype': 'm.audio',
     'org.matrix.msc1767.audio': {'duration': 68000,
                                  'waveform': [0,
                                               986,
                                               408,
                                               646,
                                               0,
                                               663,
                                               833,
                                               68,
                                               408,
                                               969,
                                               34,
                                               0,
                                               476,
                                               255,
                                               0,
                                               697,
                                               340,
                                               272,
                                               0,
                                               0,
                                               374,
                                               0,
                                               102,
                                               0,
                                               374,
                                               85,
                                               442,
                                               0,
                                               476,
                                               0,
                                               0,
                                               85,
                                               272,
                                               0,
                                               204,
                                               238,
                                               476,
                                               0,
                                               170,
                                               0,
                                               442,
                                               119,
                                               289,
                                               0,
                                               0,
                                               476,
                                               238,
                                               595,
                                               0,
                                               0,
                                               391,
                                               0,
                                               0,
                                               646,
                                               374,
                                               238,
                                               0,
                                               0,
                                               0,
                                               0,
                                               0,
                                               799,
                                               34,
                                               0]},
     'org.matrix.msc3245.voice': {},
     'url': 'mxc://xxx.xxx/PFsnoEkSeCsjQgngRdprpoeS'}

Any idea on how to fix this?

Thanks in advance!

hibobmaster commented 10 months ago

Can you give me a simple step how to reproduce it? Where did you get it?

rhetenor commented 10 months ago

I've cloned the code and setup a virtualenv since I don't have Docker at my hand. Afterwards I followed the instructions from the README and Wiki configuring everything (including room encryption keys) and run the bot as stated in the Dockerfile with python src/bot.py.

I then tried the bot first with a test voice message through a direct message and afterwards I setup a separate new room invited the bot and sent the message to this room. It always outputs the snippet I've pasted onto the console.

My python version is: 3.9.2 and my matrix-synapse version is 1.96.

hibobmaster commented 10 months ago

Have you installed libolm-dev dependence?

rhetenor commented 10 months ago

Yes, they are installed.

hibobmaster commented 10 months ago

image

This is my information got from element web when i debug the e2ee room. Original event source:

{
  "type": "m.room.encrypted",
  "room_id": "!JgRZxdctiWOVQdqbBO:matrix.qqs.tw",
  "sender": "@bobmaster:matrix.qqs.tw",
  "content": {
    "session_id": "OVyj5G3L4ObVBtAM2SeOcKB1OVM2lq2euij+fkYUKVc",
    "ciphertext": "AwgIEuAF3d18JhBzuM/TO41irlJ+IZEKhRi6KG0JQtTJBHr8HCcm6hiQ0HcsPiHSOq9xDGgkzFmvMemsuTo9GbolsXwq1Y2BZPOy5E8roJEvH3dtKSyM72TirPNTr0omEyK/iaowYvEn0YKzmIXPFBqYNzYcY3zbR+LcRhPZbKLBmDRsybwprzBhv0R0onUWNNBB89iCLww0XTf31FK2cn1YuoiYlhxoZoWkK3XxoxH17MThi0SSsoVWSF1OOwL8fNEixTf61pbnBYxeHitj0EWoqDobSnu6Y2bc+077Ij37qsAatTw0wt5WJfXvCo7i1O/LzgPdSD0mBbqTpwESJt8MZ5/TKcpRaXwh7FCTGTw8oHZ3dcFD11rxmSnDBYZdGALWkGdSEa6yz+uXvWaE06MnDGG0cgShCGT+TbZvZuc+Lm4GnYQiAuaUXEFmsIFtKhNoyJmbfoHTSn8z/I1TWau2/hZLvRhKgALqik9VHBXOLXoXopVB/TveHTG4sTkCHOrQGjZqtcH8VrWmGh1iwR1OHSjAEQJvKSbEniZP5lv0pCwn15kKkKCxbAPNNNkTNBnMt17bIcZb86/zcwPbXxxmdthaTpZrV4zp7A/Hj2zf1h96W5EDz3cgreE28uMsWgR4RNm8Sgzg8+d001pHop8bA3cGXIEfQAXjOOWISkOJKyScUZ2Ylaxuis8cScWUKaU8byCb03NKYYHN+zdQqVBs53L7poee82ZdemAQ75hlKDpwuo0KHpvb9fVazLhKv840k5EJwy4X3lw81JZHbrl6NqZDdTda+rW6/eH4aaH+b5mMuwABY0i6sIAWj20Z7g+E9p90waQzvkmYDbqKJanqCgKdrl/JSmATm3yXU4Ea2BdJhBniOmqeXlBEuq9LqZ6wuP6SGF2GV4GmimWdmoXg7whABzZUg3+gYDxk6JqTiMNoBv1ti/UAaB5gp5dMC26TJY4AmXrEFmgrwg3jbs95NS96FtqIAHlnZYFFmNcbhjYSySY+gRRF1Nq1Jl302AZ9/6wGJ6BOFXqpoUJKBnd+VDyhCcHGPEfozfJFkG0/1Wxem8DlMFsjBNQ4Cw",
    "device_id": "KEFILXMLSH",
    "algorithm": "m.megolm.v1.aes-sha2",
    "sender_key": "iR2WbpuGP+5hiK6QvA/bfvOygLHfzK220gBr27MUy1k"
  },
  "origin_server_ts": 1701767768238,
  "unsigned": {
    "age": 3346377837
  },
  "event_id": "$hx_xqfqWthKS7hghUwnuuTzE3tPF8ZuSX58aleWmNW4",
  "user_id": "@bobmaster:matrix.qqs.tw",
  "age": 3346377837
}

Decrypted event source:

{
  "content": {
    "body": "Voice message.ogg",
    "file": {
      "hashes": {
        "sha256": "y31wS0vILW2REC6X9JY6UdT431FYhP/j7m9i0B9p5ls"
      },
      "iv": "TtaDN2FA+FsAAAAAAAAAAA",
      "key": {
        "alg": "A256CTR",
        "ext": true,
        "k": "k81UlnN3QwEiG3XtKLoLCxZ1fzIuvzz7xW4P06hi8s0",
        "key_ops": [
          "encrypt",
          "decrypt"
        ],
        "kty": "oct"
      },
      "url": "mxc://matrix.qqs.tw/2a71e736a392defdb2258519b3ee9e3bbf0b75cf1731965657437700096",
      "v": "v2"
    },
    "info": {
      "duration": 1594,
      "mimetype": "audio/ogg",
      "size": 6858
    },
    "msgtype": "m.audio",
    "org.matrix.msc1767.audio": {
      "duration": 1594,
      "waveform": [
        0,
        0,
        0,
        1,
        24,
        36,
        39,
        40,
        101,
        106,
        66,
        75,
        121,
        77,
        63,
        84,
        97,
        247,
        182,
        999,
        1024,
        471,
        785,
        772,
        673,
        569,
        714,
        803,
        708,
        428,
        230,
        148,
        618,
        81,
        144,
        0
      ]
    },
    "org.matrix.msc3245.voice": {}
  },
  "room_id": "!JgRZxdctiWOVQdqbBO:matrix.qqs.tw",
  "type": "m.room.message"
}
rhetenor commented 10 months ago

I messed up with the device ID, now it works! The error confused me and I was looking into the wrong things.

Thanks so much for your help! Hopefully I never have to listen to voice messages again! :)