OMEMO in Conversations client doesn't works out of the box

[imShara]

What version of ejabberd are you using?

Ejabberd 18.04-1

What operating system (version) are you using?

Linux archlinux 4.16.9-1-ARCH

How did you install ejabberd (source, package, distribution)?

Pacman repository

What did not work as expected? Are there error messages in the log? What was the unexpected behavior? What was the expected result?

OMEMO Encryption doesn't work out of the box in Conversations client. No key sharing, message can't be send.

Logcat error:

I/conversations(30616): AxolotlService (@.): Creating axolotl sessions if needed... D/conversations(30616): AxolotlService (@.): Finding devices without session for admin@ovel.space W/conversations(30616): AxolotlService (@.***): Have no target devices in PEP!

[zinid]

grep OMEMO ejabberd.yml maybe?

[imShara]

@zinid, yep.

Should comment setting in force_node_config to make OMEMO work

force_node_config: 
  ## Avoid using OMEMO by default because it
  ## introduces a lot of hard-to-track problems
  ## "eu.siacs.conversations.axolotl.*":
  ##   access_model: whitelist

But it so unclear and no docs how to setup it properly

[aiomaster]

Can someone please clarify what this means:

it introduces a lot of hard-to-track problems

What kind of problems can occur? I've never heard of one.

[weiss]

What kind of problems can occur?

Many clients don't support it; e.g., none of the existing web clients I'm aware of currently do. Other clients don't support it properly; e.g., not in group chat or not with non-contacts. Other clients require manual installation of some plugin. Other clients don't hide key verification from users (or even enforce verification), so some users will just give up because they have no clue what they're supposed to do; others will verify keys and thereby break communication with unverified devices, e.g. when the contact buys a new phone. Due to forward secrecy, archives aren't accessible from new devices or from an occasionally used web client. Also, OMEMO relies on PEP and (to also work with non-contacts) on some optional PubSub feature which not all servers support (properly), so if such a server is involved, this can lead to additional breakage.

Each of these issues is a UX nightmare, IMO. Power users might be able to cope with some of them, but ordinary mortals will just give up.

[aiomaster]

Hello @weiss, thanks for you thoughts. In my personal opinion I don't think it's that worse. More and more clients support OMEMO or plan to support it in the future: https://omemo.top/ The legacy OTR is dropped from more and more clients instead. And not every Client is a UX nightmare. Conversations does it very well in my opinion. Key verification is a central part in that protocols and if you don't do it or can't do it, you have just less security and you should know it. Security is never for free and users have to understand the implications if a tool claims that.

The thing I just don't understand is, why the example configuration of ejabberd explicitly disables it. If someone don't want to use it, its fine, but it doesn't seem to break something for users if it is just not disallowed. In other words: I don't understand the logic in: "The UX implementation in some clients is bad, so better block it per default for all users." Or do I miss something here?

For me it is not a big thing, cause I can simply comment it out, but I just want to understand the reasons.

[zinid]

I don't understand the logic in: "The UX implementation in some clients is bad, so better block it per default for all users."

And what's hard to understand here? We don't want XMPP users to have bad experience. Also note that you can always enable OMEMO, but you cannot disable it because this will be treated as a downgrade attack by a client.

[aiomaster]

The funny thing is: I switched from prosody to ejabberd and my users (~50) had a bad experience cause OMEMO wasn't working out of the box and I had to find these lines to comment it out and make it work again. I really don't get, why the server software should take care of the UX - this should be the responsibility of the clients. The server should just work and don't block something, cause it is thinking, that the clients are bad. But hey, to make a constructive suggestion: You can add to the comment "If you want OMEMO to work properly, than you should comment out the following lines". Thanks for your replies!

[zinid]

The server should just work and don't block something, cause it is thinking, that the clients are bad.

I disagree. The server might have policy which clients should respect. Since there is no policy "OMEMO is not allowed" we use this hack.

[aiomaster]

Wow this was fast. I still not agree that OMEMO should be blocked by default, but the new comment helps to know, what to do to enable it. Thanks all for your statements and thanks to @imShara for the commit.

[imShara]

@aiomaster, Anyway, everyone will make init configuration. It's nice when default config is minimalistic and you need to enable what you want. The problem was only in unclear description, which messed me at first ejabberd configuration.

[weiss]

Key verification is a central part in that protocols

Yes, if you use XMPP for highly sensitive communication (which I'm not sure is a good idea) and fear being subject of a man-in-the-middle tampering with your communication, then key verification makes sense. So if clients want to support this use-case, they should offer verification. But IMO this should be well hidden in some expert settings. Normal people who just want to use XMPP for day-to-day communication just won't be able to cope with it. As soon as the contact of a ChatSecure user buys a new phone, messages sent to him are silently lost and the ChatSecure user will have no clue how to fix it without asking some geek.

Security is never for free and users have to understand the implications if a tool claims that.

The users I have in mind won't care at all about what you think they should understand. They'll switch back to WhatsApp because our crap just doesn't work.

[arendtio]

@weiss You do know that conversations switched to TOFU a few months back? Since then it is very easy to use. Probably even too easy as I am not sure how secure that model is now.

Nevertheless, I just had a user walk over to me and ask why she couldn't send a message. The reason was simple: Today, I migrated my old config to the current example config and saw that comment and thought 'oh cool, I don't even have to remove the '#' they seem to support OMEMO out of the box'. I didn't notice that you boycott OMEMO out of the box :-1:

Regarding hard to use: I have a few 50+ Users on my server which are neither hardcore XMPP users nor trained in IT personnel and yet they somehow manage to use OMEMO on a daily basis. In my opinion, that whole paragraph should be removed from the example config as it does no good there anyway.

[licaon-kter]

@arendtio Because you misunderstood what is written? Not sure that's useful. You or the admin need to know what you deploy.

[arendtio]

@licaon-kter sure, I am responsible for what I deploy, but that is not the point. I found it quite counterintuitive and I find the reasoning behind it wrong. I mean, that default config explicitly disables the most advanced end-to-end encryption we have in XMPP...

[licaon-kter]

@arendtio ...supported fully by how many clients exactly? Right, advanced, yes, trouble free on Conversations only...

[zinid]

Guys, the issue tracker is not the place for a drama, please chill out or I will have to mute the thread.

[weiss]

@arendtio

You do know that conversations switched to TOFU a few months back? Since then it is very easy to use.

You mean Bilnd Trust Before Verification (BTBV), which is a bit older. BTBV works as long as you (1) don't use the (now prominently exposed) barcode scanner for adding contacts and (2) everyone uses Conversations. However, ejabberd is an XMPP server, not a Conversations server.

[arendtio]

@weiss yes, I mixed up TOFU and BTBV, but that shouldn't make a big difference.

Nevertheless, I think I made my point and I would like to respect the warning of @zinid

[paradisaeidae]

Hi, Ejabberd I've been using for some years, very well indeed. Now running 18.09 on Ubuntu 18.04. Generally I've made adjustments as needed. Recently I've been testing OMEO with a buddy who uses chatsecure/ios. Some two months back I was using Gajim on Linux. For some reason I had to change the config. Cannot remember exact scenario, sorry. Though when I did Conversations would not get new set of keys. Server issue. When I looked into the Mnesia tables key entries were there. I tried deleting them. This was not effective. Tried some other techniques, delete user on Conversations client, delete on server, recreate. Still not effective. The only clearance that had effect was to rebuild the entire Mnesia DB.

Recently I switched to a Win10 box running Gajim. When I turn on omemo I see: No devices found. Query in progress... Which is probably a good thing. With Conversations I see Error: There are no usable keys available for this contact. Fetching new keys from the server has been unsuccessful. Maybe there is something wrong with your contact's server. Now I see a stanza in the log:

<iq xml:lang='en' to='some@one/phone' from='some@one.else' type='error' id='mUoF15zakKXTNw'>
    <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <items node='eu.siacs.conversations.axolotl.devicelist'/>
    </pubsub><error code='405' type='cancel'>
    <closed-node xmlns='http://jabber.org/protocol/pubsub#errors'/>
<not-allowed xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/></error></iq>

A bit back in the log I see:

  sub_els =
        [#pubsub{
             subscriptions = undefined,subscription = undefined,
             affiliations = undefined,publish = undefined,
             publish_options = undefined,subscribe = undefined,
             unsubscribe = undefined,options = undefined,
             items =
                 #ps_items{
                     xmlns = <<>>,
                     node = <<"eu.siacs.conversations.axolotl.devicelist">>,
                     items = [],max_items = undefined,subid = <<>>,
                     retract = undefined},
             retract = undefined,create = undefined,configure = undefined,
             default = undefined,delete = undefined,purge = undefined,
             rsm = undefined},
         #stanza_error{
             type = cancel,code = 405,by = <<>>,reason = 'not-allowed',
             text = [],
             sub_els = [#ps_error{type = 'closed-node',feature = undefined}]}],

So it appears to be a lack of affiliations? Even though we have 'both' pub-sub, this is not yet applied to this node?

I understand OMEMO is not on the wish-list of many users, though it is being included in some good clients. So people will press the button, just to see.

Thanks!

[licaon-kter]

@paradisaeidae config?

[paradisaeidae]

Which section of ejabberd.yml is relevant?

[zinid]

@paradisaeidae mod_pubsub's options.

[paradisaeidae]

Thanks Zinid, Not sure what to change:

mod_pubsub:
  access_createnode: pubsub_createnode
  ignore_pep_from_offline: false
  last_item_cache: false
  max_items_node: 1000000
plugins: 
    - "flat"
    - "hometree"
    - "pep"
  force_node_config:      ## Avoid using OMEMO by default because it introduces a lot of hard-to-track problems
  "eu.siacs.conversations.axolotl.*":
      access_model: whitelist       ## Avoid buggy clients to make their bookmarks public
    "storage:bookmarks":
      access_model: whitelist

[licaon-kter]

@paradisaeidae Look at the same section in the updated example config: https://github.com/processone/ejabberd/blob/master/ejabberd.yml.example

[zinid]

Seems like there is also a mess with indentation.

[paradisaeidae]

Changed to this: Found the backtick+yaml for markdown technique!

  mod_pubsub:
    access_createnode: pubsub_createnode
    ignore_pep_from_offline: false
    last_item_cache: false
    max_items_node: 1000000
    plugins: 
      - "flat"
      - "hometree"
      - "pep"
    force_node_config:      ## Avoid using OMEMO by default because it introduces a lot of hard-to-track problems
      "eu.siacs.conversations.axolotl.*":
        access_model: open       ## Avoid buggy clients to make their bookmarks public
      "storage:bookmarks":
        access_model: whitelist

[licaon-kter]

Those comments are kinda on the wrong rows, it doesn't matter for ejabberd but you might find them confusing in the future

[paradisaeidae]

Hi, OMEMO is the adventure! Adjusted config to be pubsub open. Now is see this in debug:

    sub_els =
        [#pubsub{
             subscriptions = undefined,subscription = undefined,
             affiliations = undefined,publish = undefined,
             publish_options = undefined,subscribe = undefined,
             unsubscribe = undefined,options = undefined,
             items =
                 #ps_items{
                     xmlns = <<>>,
                     node = <<"eu.siacs.conversations.axolotl.devicelist">>,
                     items = [],max_items = undefined,subid = <<>>,
                     retract = undefined},
             retract = undefined,create = undefined,configure = undefined,
             default = undefined,delete = undefined,purge = undefined,
             rsm = undefined},
         #stanza_error{
             type = cancel,code = 405,by = <<>>,reason = 'not-allowed',
             text = [],
             sub_els = [#ps_error{type = 'closed-node',feature = undefined}]}],
    meta = #{ip => {1,129,104,123}}}

Looks like this is from: https://github.com/processone/ejabberd/blob/d79ddd7b5c0a96c478df5a130cd62e43ef30c1db/src/mod_pubsub.erl#L1689

This is all I know today.

[licaon-kter]

@paradisaeidae Explain the log... what client... what did you do... etc...

[paradisaeidae]

My apologies, My head is full, pretty close. This is result of attempting to gain keys using gajim on Win10. Gajim attempts to get keys reports: 'No devices found. Query in progress...' Have restarted ejabberd, yep. Previously on a fresh install of ejabberd keys are gained ok. Changes of client seems hazardous. Conversations reports fetching error: "There are no usable keys available for this contact. Fetching new keys from the server has been unsuccessful. Maybe there is something wrong with your contact's server." Would I have to delete the nodes for the new config to be applied? I have tried doing this previously. There is a mnesia table editor I used. Though that was not effective, possibly due to my lack of understanding.

[paradisaeidae]

I am slowly understanding OMEMO and it's implementation mechanics. Have had some days break. Now it is clear that there are no keys for the client to read. If omemo requires end-to-end key exchange then the peer target being offline will be an 'issue'. So a more meaningful response to the request would be something which states this?

Interestingly the 'not-allowed' phrase is here: https://github.com/processone/ejabberd/search?q=not-allowed&unscoped_q=not-allowed Which is in the muc code. I am yet to understand why.

Could it be that ejabberd is not (able) not send a key request to the peer target device? I don't see attempts to do this in the ejabberd debug log. Though I do see the peer device (chatSecure) providing key in preKeyPublic preKeyId='37' type stanzas. I see ejd supplying response stanzas with type='result' to the chatSecure client.

More soon!

[tallero]

Just wanted to share the fact that on my ejabberd server (hosted on a vmicro machine) OMEMO key exchange between contacts can be very slow, requiring from one minutes to three days for a client to see other contact's key the first time, while key exchanges between one own clients, 2 gajim and 1 conversations for example, can take weeks or never happen in general.

Months ago both gajim AND conversations had a bug that made them not showing at all incoming encrypted messages they did not have key for (this did not happen for a while though), thus breaking sync and availability.

This tended to become very common when one used more than two devices to connect.

Also because on how OMEMO works I believe (I am referring to the 100 keys rule) one has to wait for key syncing all over again when you do not use one of your device for long time.

Apart from slow key exchange, one-device accounts support is really good and configuration tends to be a one-timer.

This was to say that I can see why this is not enabled by default.

EDIT: I started using OMEMO on ejabberd 16 and I did not have the config line this issue was about, so I did not even know of this.

@paradisaeidae In general no one of my subscribers can see OMEMO keys immediately. Generally they can when they have pubsubbed each other and have chatted for some time.