gajim-omemo to gajim-omemo do not exchange new keys / devices / fingerprints; sending message fails

[silberzwiebel]

I'm running gajim 0.16.5 on Fedora 24 with the latest gajim-omemo code from this repository. My contact runs gajim on Linux Mint Debian Edition (not sure which version) with the gajim-omemo plugin from the plugin installer inside gajim. When she installed the plugin, I thought it should be possible to exchange encrypted messages over gajim (we both use also conversations which works fine with OMEMO but I also do not receive the new device key in conversations but I guess I should file another bug for this, shouldn't I?).

However, my gajim does not receive her new device key / fingerprint. This was with a version from some days ago so I updated my gajim-omemo and now get actually an error in the GUI (but not in the debug log, btw) when I enable OMEMO and then try to send a message:

Traceback (most recent call last):
  File "/usr/share/gajim/src/chat_control.py", line 801, in _on_message_textview_mykeypress_event
    self.send_message(message, xhtml=xhtml) # send the message
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/ui.py", line 148, in omemo_send_message
    self.new_fingerprints_available()
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/ui.py", line 193, in new_fingerprints_available
    fingerprints = self.state.store.getNewFingerprints(self.contact.jid)
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/omemo/liteaxolotlstore.py", line 97, in getNewFingerprints
    return self.identityKeyStore.getNewFingerprints(jid)
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/omemo/liteidentitykeystore.py", line 152, in getNewFingerprints
    for row in c.execute(q, (jid,)):
OperationalError: no such column: shown

Apparently, the plugin tries to get the new device key but fails to do so?

In the debug-log there is another error in the beginning:

03.08.2016 22:54:59 (D) gajim.plugin_system.omemo:  jabberserver.tld  => 100 PreKeys available
03.08.2016 22:54:59 (D) gajim.plugin_system.omemo:  jabberserver.tld  => Adding OMEMO ui for contact@jabberserver.tld
03.08.2016 22:54:59 (W) gajim.plugin_system: Error executing <bound method OmemoPlugin.connect_ui of <gajim-omemo.OmemoPlugin object at 0x7f5a16025910>>
Traceback (most recent call last):
  File "/usr/share/gajim/src/plugins/pluginmanager.py", line 259, in _execute_all_handlers_of_gui_extension_point
    handlers[0](*args)
  File "/usr/share/gajim/src/plugins/helpers.py", line 107, in wrapper
    result = f(*args, **kwargs)
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/__init__.py", line 378, in connect_ui
    self.ui_list[account][contact_jid].new_fingerprints_available()
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/ui.py", line 193, in new_fingerprints_available
    fingerprints = self.state.store.getNewFingerprints(self.contact.jid)
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/omemo/liteaxolotlstore.py", line 97, in getNewFingerprints
    return self.identityKeyStore.getNewFingerprints(jid)
  File "/home/thomas/.local/share/gajim/plugins/gajim-omemo/omemo/liteidentitykeystore.py", line 152, in getNewFingerprints
    for row in c.execute(q, (jid,)):
OperationalError: no such column: shown
/usr/lib/python2.7/site-packages/nbxmpp/tls_nb.py:436: DeprecationWarning: unicode for cipher_list is no longer accepted, use bytes
  tcpsock._sslContext.set_cipher_list(self.cipher_list)
03.08.2016 22:55:04 (D) gajim.plugin_system.omemo: jabberserver.tld => Announce Support after Sign In

Server is self-hosted prosody 0.10 nightly. Thanks for your efforts!

[lovetox]

sorry that was my fault i added a column and didnt do another dbmigration its easy solveable though connect to the sqlite DB its called "yourserver_omemo.db"

and set the prama user version to 4, and restart

or run this query on your db

ALTER TABLE identities ADD COLUMN shown INTEGER DEFAULT 0;
                UPDATE identities SET shown = 1;

_DONT DO BOTH_

this problem is cause you where on some in between git version and not stable

[lovetox]

and i updated the readme, the workflow for receiving keys and starting encryption is now a bit different (easier though)

https://github.com/omemo/gajim-omemo/blob/master/README.md

please upgrade your contact to 0.8 and check if the problems still exist

[silberzwiebel]

connect so the sqlite DB its called "yourserver_omemo.db"

and set the prama user version to 4

thanks for the quick reply. I'm not familiar with sqlite, however. Could you tell me in more detail what I should do? Thanks.

[lovetox]

im not familiar with fedora, but is there not a package for sqlite in your package managment?

on ubuntu i do sudo apt-get install sqlite

[silberzwiebel]

I do have a sqlite3 executable and this works to open the .db file. But I do not know what commands I would need to type in there to actually do the changes required...

[lovetox]

try this

first: ALTER TABLE identities ADD COLUMN shown INTEGER DEFAULT 0;

then:

UPDATE identities SET shown = 1;

[silberzwiebel]

okay, thanks, the error on my side is gone (sorry, I only saw later that you edited your comment above with the sql commands ;)) The fingerprint however is not received. I saw you released 0.8 very recently. When is 0.8 available trough the plugin installer? Because I do not want to put the hassle on cloning a git repo to my contact... (and doing this breaks thing as I'm experiencing myself :-/)

[lovetox]

probably on the weekend, check your log everything is there

if you dont have this line in there

Received device list for friend@yourfriend.com:[1391233980]

then this means you friend has not published any devices. does your communication work if you using conversations?

[lovetox]

on what server software does your friends server run? is he online at the time? with the new version 0.8 you have to try sending a message to get the keys, actually write something and press the send button after that look at the log what does it say?

[silberzwiebel]

interestingly, I only get the

Received device list for ...

line for three of my contacts but not the one I'm having this current problem with. I do, however, successfully communicate with more than three contacts via OMEMO and Conversations. OMEMO does also work with no problem for the contact in question here (but: Conversations also does not fetch the new device coming from gajim-omemo).

All contacts are on the same server. She was online when we tried to use OMEMO. I will report back once she has installed 0.8 via the plugin installer.

[lovetox]

devicelists are send to you when you come online.

if the server of your contact dont, there are only 2 possible options

1. client didnt announce (you can also see this in the log, even if it was successful)
2. he did, and the server is not publishing it to the other contacts

really the log shows you everything, every step and where something goes wrong

[Yakulu]

I got a similar problem since 0.8 update. Everything was working fine in 0.7.x. I tried SQL commands without success. I talk with a friend from Conversations to Conversations without trouble. It's just Gajim-OMEMO <-> Conversations that have a problem, not accepting sending message nor receiving one. My DEBUG trace (anonymized) :

08/08/2016 09:20:25 (I) gajim.plugin_system.omemo xxx.net => Received device list for name@xxx.net:[289207431] 08/08/2016 09:20:25 (I) gajim.plugin_system.omemo xxx.net => Saved devices for name@xxx.net 08/08/2016 09:20:25 (I) gajim.plugin_system.omemo xxx.net => Publishing bundle was successful 08/08/2016 09:20:25 (I) gajim.plugin_system.omemo xxx.net => Devicelistquery was successful 08/08/2016 09:20:51 (I) gajim.plugin_system.omemo xxx.net => Missing device sessions for name@xxx.net: [289207431] 08/08/2016 09:20:51 (I) gajim.plugin_system.omemo xxx.net => Fetch bundle device 289207431#name@xxx.net 08/08/2016 09:20:51 (D) gajim.plugin_system.omemo xxx.net => No Trusted Fingerprints for name@xxx.net 08/08/2016 09:20:51 (E) gajim.c.ged Error while running an even handler: <bound method OmemoPlugin.handle_iq_received of <omemo.OmemoPlugin object at 0x7f490ced2d50>> Traceback (most recent call last): File "/usr/share/gajim/src/common/ged.py", line 93, in raise_event if handler(_args, _kwargs): File "/usr/share/gajim/src/plugins/helpers.py", line 107, in wrapper result = f(_args, _kwargs) File "/home/fabien/.local/share/gajim/plugins/omemo/init.py", line 446, in handle_iq_received iq_ids_tocallbacks[id](event.stanza) File "/home/fabien/.local/share/gajim/plugins/omemo/init.py", line 476, in device_id) File "/usr/share/gajim/src/plugins/helpers.py", line 107, in wrapper result = f(_args, *_kwargs) File "/home/fabien/.local/share/gajim/plugins/omemo/init.py", line 518, in session_from_prekey_bundle if state.build_session(recipient_id, device_id, bundle_dict): File "/home/fabien/.local/share/gajim/plugins/omemo/omemo/state.py", line 99, in build_session sessionBuilder.processPreKeyBundle(prekey_bundle) File "/usr/lib/python2.7/dist-packages/axolotl/sessionbuilder.py", line 142, in processPreKeyBundle raise InvalidKeyException("Invalid signature on device key!") InvalidKeyException: Invalid signature on device key!

[vanitasvitae]

I do have the same problem @silberzwiebel described in this issue. I cannot run the sql queries successful though: Im on debian 8.5 stable. When I use sqlite myserver_omemo.db and enter the first command, sqlite prompts me with a error:

sqlite> ALTER TABLE identities ADD COLUMN shown INTEGER DEFAULT 0;
SQL error: near "ALTER": syntax error

When I use sqlite3 instead of sqlite I get the following error:

sqlite> ALTER TABLE identities ADD COLUMN shown INTEGER DEFAULT 0;
Error: no such table: identities

What can I do?

[vanitasvitae]

Nevermind, I fixed it. I had opened the wrong database. For me it was omemo_myserver.db, not the other way round.

[lovetox]

so @vanitasvitae everything working now?

@Yakulu do you have the same traceback after every restart if you try to send him a message?

does this problem exist only with this contact or with others too?

this seems like he has published a faulty signature for some reason. maybe tell him to log out and sign in again with his conversations account.

can you look at the log after he send you a message? you receive every message but it probably cant be drecypted, so the error would be interesting.

[Yakulu]

@lovetox : thank you for your reply. My contact tried reconnect without success. I have no problem with a new contact today (but I've not many OMEMO ready friend list...).

About when the buggy contact send a message :

08/08/2016 16:47:53 (D) gajim.plugin_system.omemo xxx.net => OMEMO msg received
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo ---------------
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo <message xmlns="jabber:client" to="fabien@xxx.net" type="chat" id="32b039b9-0862-4061-be39-72688735cbd1" from="name@xxx.net/phone"><encrypted xmlns="eu.siacs.conversations.axolotl"><header sid="289207431"><key rid="1909214104">MwohBRyMV2t1E3wuTxhLBr7kmzq9Gg4lNL5+ulvu2+xufHMfEAAYASIgdag6IB5Mkzzivcp0NXNu
9f5JyhATpDIG684r0k427tuDNtCHSDlGnA==
</key><key rid="1834206689">MwohBddrnesirtSvgJrOc/qQ+7xY9+Z8YpVHQGXRo+IHQVACEBsYGCIg5IJCkNnFlPwu+nRJAlF9
guLuKLIn8ENUYC7Iwg7yQq3KpMTItmXbfg==
</key><iv>2eZ4Wdb19jIKNYg8HgoSOg==

08/08/2016 16:47:53 (D) gajim.plugin_system.omemo </iv>
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo </header>
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo <payload>VxcIE3cp1zcvYfTxAW8gNdFbp5mAxOIKtiS2RvEXpRM7aW4476wcc6+cJ3GXfD4AE4R2mpkkyu1l

08/08/2016 16:47:53 (D) gajim.plugin_system.omemo </payload>
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo </encrypted>
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo <markable xmlns="urn:xmpp:chat-markers:0" />
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo <request xmlns="urn:xmpp:receipts" />
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo <store xmlns="urn:xmpp:hints" />
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo </message>
08/08/2016 16:47:53 (D) gajim.plugin_system.omemo ---------------
08/08/2016 16:47:53 (W) gajim.plugin_system.omemo OMEMO message does not contain our device key

[lovetox]

he sends you messages not encrypted for your gajim client.

he should look if he accepted your fingerprint in conversations, you can find your own fingerprint Omemo Icon -> Own Devices -> Own Fingerprint give him the last 4 numbers and let him check

can you tell me which server he uses? domain?

[vanitasvitae]

@lovetox: yep, its working again.

[Yakulu]

@lovetox : sorry for the noise. It worked after approving my finger in his account (it seems that I have forced generation of a new one after the problem happens : I tried reinstalling plugin with removal of local databases).

We still can communique after finger approval but all worked when he reconnects to the server. So I guess a simple reconnection after plugin update have worked too.

Thank you for your help.

[lovetox]

good to hear.

deleting the database should only be last resort measure it resets everything, generates new keys and fingerprints.

you probably have now your old fingerprints and new fingerprints published

you should go to the plugin config menu, last tab and press "clear devices" that removes old fingerprints not used anymore from the server.

you will still see them in the fingerprint tab but they will be grey. if you add new contacts they will never see them.

[Yakulu]

you should go to the plugin config menu, last tab and press "clear devices" that removes old fingerprints not used anymore from the server.

Thank you. It's done.

[lovetox]

@silberzwiebel did you resolve your problem?

[silberzwiebel]

No. My contact upgraded gajim-omemo to 0.8.1 but still I do not receive her new device key / fingerprint, neither on gajim nor on Conversations. How can I check if my server is dong things wrong, i.e., not distributing the keys? Because in the logs I get the "Received device list for ..." only for a few contacts, although I'm using OMEMO succesfully with more contacts using Conversations (again, all on the same server).

[lovetox]

did you try to remove and add the contact again to your contact list? it could be a problem with subscription. if you right click in gajim on the contact - Manage Contact -> Abonnoment: both options have to be grey

no matter what you see, try to readd the contact.

to see if the contact correctly publishes his device id, look at the logs if the contact also uses gajim. if the publishing doesnt work, there must be errors in the beginning

[silberzwiebel]

I tried to re-add but it didn't help. I will look at the logs of my contact presumably at the weekend.

[lovetox]

did you get a look at the logs?

[silberzwiebel]

No, because it suddenly worked. It seems there is something wrong with the roster management in gajim and / or Conversations. Somehow, I can click "Allow contact to see my status" all the time. When the contact was online at the same time when I clicked this on my gajim, I received her OMEMO key, in both Conversations an gajim. I do have another PC running gajim, where I did not receice the key. I'll try to use the same workaround there soon and report back.

[lovetox]

the same buttons are in conversations in the contact detail view. so you can click what you want if for example the contact hasnt ticked that box in conversations for example

both subscriptions status have to be ticked, in gajim and in conversations.

[lovetox]

as this seems to be an issue with subscription either in gajim or conversations, and not the plugin i will close this :)

feel free to reopen if you find something that points to a bug with the plugin