Support for client roster organization based on server-side Roster Groups (see RFC 6121, 2.1.2.6. Group Element)

[zippydan]

I'm running a private OpenFire server with many users divided into defined groups based on departments.

I've used the following XMPP clients that all support displaying the user list (roster) with users organized by groups, by default:

Windows: Spark Jitsi

Android: Xabber Conversations (does not organize users by group by default, but displays group tag and supports sorting by group)

macOS: Messages (native macOS app, back when it still supported XMPP pre-Mojave) Adium Jitsi Swift Converse.js Desktop

iOS: JabberB

Browser: Converse.js

Is there any way that Monal could also support this standard? (I assume it is a standard. Edit: It is part of the RFC core standard.) Right now, Monal is just showing me an overwhelming list of online users without any way to easily find the person I need.

P.S. I'm looking for a new macOS client now that Apple in their infinite wisdom decided to drop XMPP from the latest version of Messages in Mojave. :( Off to try Adium now, but I much prefer the seemingly overall less bloated approach of your app.

P.P.S. Yup, Adium supports the organization by user groups also.

[anurodhp]

I have the group info saved in the db for every contact I removed it at some point 5-6 years ago. I haven't surfaced it. I'll look at readding it

[benjaminbischoff]

One point as an iOS user, I think if XMPP hat the plan to be used from many people, on iOS it should not be actived by default! :)

[zippydan]

Don't really care if it is default on or off, as long as there is an option to show groups.

In my opinion, it should default to on, as the XMPP server admin decides whether there are groups or not, and if he decides the users should be organized by groups, there is usually a good business or organizational reason for doing so. In our case, we have so many users that the list becomes overwhelming and unwieldy without them.

[zippydan]

Any chance this will be added?

[Echolon]

Is that related to a specific XEP actually?

@zippydan I think that can be added one day, however I think Monal is fighting some more important fronts atm. But we will keep that open. Try Siskin or Beagle in the mean time. Converse.js in the webbrowser does support it I think

[zippydan]

I just want to confirm that the feature request here matches my intentions. I'm a little hung up on the word "create" since the user groups I use are "created" and defined SERVER side. The client then simply supports this central organization and DISPLAYS the server-defined groups. So, maybe this is just a misunderstanding on my part or a poor choice of words, but what I'm asking for would NOT be the ability to "create" user groups on the CLIENT side.

I could see someone wishing they could organize users on their local client, so that might be a different feature request. It's not relevant to my own use case and is not what I was originally requesting.

If we're on the same page then just ignore this. I just thought it would be better to clarify just in case.

[Echolon]

@zippydan okay, sorry. the I mistook it. May you try to edit the title and first description again?

[Echolon]

And just to say the prios in Monal are at the moment fighting bugs and some other things. So no expectations when this might be implemented

[tmolitor-stud-tu]

I might implement this as tags below the contacts like conversations does this (readonly support, like conversations, but your usecase only requires read support anyways)

[tmolitor-stud-tu]

tapping onto such a tag would filter the list by this tag

[zippydan]

@tmolitor-stud-tu

So, I took a look at Conversations (on Android) and you're right that they handle the server-defined user groups as a tag under each user name, and then don't bother actually using the tag for any sorting.

I'd ask you to take a look at the way Xabber (on Android) handles group sorting instead, as I find this much more useful to my use case. I could see the Conversations approach being useful for someone with a small group users/friends, or for someone who is using xmpp outside of a business environment where you don't have people naturally compartmentalized by location or department (of course now we'd get into another discussion which would be client-defined groups/tags instead of server-defined groups/tags, but I digress).

I would guess that the most useful solution for almost any user's use case would be two different client options:

1. Display Server-Defined User Groups/Tags
2. Sort by Server-Defined User Groups/Tags

Selecting option 1 alone would behave like Conversations, whereas selecting option 1 and option 2 together would behave like Xabber.

An even more robust solution would be the following options:

1. Display Server-Defined User Groups/Tags
2. Display Client-Defined User Groups/Tags
3. Sort by:
   3A. Client-Defined User Groups/Tags
   3B. Server-Defined User Groups/Tags
   3C. User Name

Here's why not displaying server-defined user groups/tags at all (as Monal currently does) is problematic:

1. Imagine, as a hypothetical, that I have 100+ users in my company, spread across 5 different sites (offices). Imagine I had a user "Maria" in two different offices. WIthout user groups, how do I know which Maria I want to contact?
2. Imagine another hypothetical: I need to contact someone in the "Engineering" department, but I can't remember their name (or the specific person is not important - I just need someone in Engineering). Without user groups, I have to scroll through 100+ names, hoping I recognize one, whereas with user groups I would just scroll to the "Engineering" heading.

Now here's why displaying server-defined user groups/tags without sorting by them (as Conversations does) doesn't really solve my usage problems:

1. Imagine the same hypothetical above, where I have 100+ users but only 3 in "Engineering". If I need to find someone in Engineering, even with group tags displayed, how far am I going to have to scroll before I happen to run across one of 3 "Engineering" tags in a list of 100? What if only 1 person from "Engineering" is online? That's incredibly inefficient and impractical.
2. Imagine I'm the "boss" and I want to see, at a glance, who is online in each office. I have to manually find a user, in each of the 5 offices, with the appropriate tag, and click on the tag in order to filter by that office name. I'll have to do this 5 times in order to "survey" the status of each office. That's a lot more work then simply scrolling through a list that is already organized by office.
3. Imagine there is an office or department, like "Engineering" above, with no one online. Then there is not even a tag available for me to click/tap on. With a sort by groups option, I can quickly see that there is an "empty" group (with no one online), but with your proposed solution, I'd either have to scroll through the entire list of 100+ users to confirm no one from that office is online, or I'd have to turn on a "show offline users" option, which requires even more steps, and just makes the roster list even more overwhelming and unusable.

[zippydan]

I'd motion to reconsider that this is a minor issue. I can't speak for every business, but for any business of sufficient size, a lack of roster organization makes Monal pretty much useless compared to any XMPP client which does support roster organization by server-defined user groups - which is a shame since Monal has exactly the kind of simple, no-nonsense interface we'd love to use for our macOS clients in our business environment.

[Echolon]

@zippydan yes, it's not that we think it is not useful, we just need to prioritize things. No one will use Monal if the urgent issues are out there, like no group chat support.

The label minor simply means the develops will first focus on other things. Those are for example listed in #322 .

If you want to increase the likeiness of having this realised you can test and review the beta if you want. Or creating attention to other develops or even try to organise funding / bounties?

maybe @tmolitor-stud-tu can estimate the needed hours for this.

btw what xep is the basis for this?

[Echolon]

https://xmpp.org/extensions/xep-0083.html

?

[zippydan]

@Echolon Forgive me if I'm out of line here, but XEP 0083 is only an optional extension for defining a unique delimiter string which would then allow for clients to optionally support nested Roster Groups.

My concern is that Monal doesn't seem to support Roster Groups at all. This is not an issue related to XEP as Roster Groups are found defined in the core specifications for XMPP, in RFC 6121 under

2. Managing the Roster -> 2.1. Syntax and Semantics -> 2.1.2. Roster Items -> 2.1.2.6. Group Element

Roster Groups (top level Roster Groups, not the nested Roster Groups defined by XEP 0083) are thus a core, default element of the 'jabber:iq:roster' namespace. In contrast, the 'roster:delimiter' option defined in XEP 0083 is part of the 'jabber:iq:private' namespace.

I could be wrong about some detail here, as I'm not an expert in this field and I'm just reading through the docs, but this does jive with the impression I've acquired from Roster Group support being present in pretty much every other XMPP client I've used. As the element seems to be part of the core XMPP data schema, I would think it's expected that client support would also not be optional (though, of course, it is not obligatory for the element for any particular user to be non-null).

You mention group chat being a bigger priority, and far be it from me to tell you how to prioritize your own program, which you generously provide free of charge, but in terms of XMPP standards, I would think supporting RFC specifications should trump XEP specifications (I'm assuming group chat is XEP 0045)?

[Echolon]

We know it is not supported. I was simply asking for the technical docs.

Which clients do support this feature?

[zippydan]

@Echolon Basically every client I've tried on every OS supports Roster Groups, every client displays Roster Groups information, and every client (except one) organizes the roster list by Roster Groups by default, other than Monal. See my original parent post for a list. (Edit: I tried out two of the clients you recommended above and Beagle IM also does not support Roster Groups. Converse.js, however, does. I don't have an iOS or iPadOS device handy so I can't try Siskin IM, but considering it has the same dev as Beagle IM, then it probably doesn't.)

Forgive my passion on this subject - I've been frustrated since first posting this issue by the feeling that we haven't been on the same page regarding what the problem is (see numerous title changes and references to other unrelated issues throughout). I thought this was an obvious issue from the start, but some of the frustration may have been due to my own inability to clearly express what was lacking.

Specifically I've been frustrated because it seems you (read: moderators and/or developers) have been treating this as a "minor", esoteric fringe issue that only affects a very small segment of use cases, when, in fact, Roster Groups are part of the core, default, basic functionality of the XMPP standard.

I'm fine with you deciding "this is not our priority" - that's totally your right - but I'm not fine with you being under the impression that this is some uncommon, highly personalized customization or feature request. The commonly used and open-source Openfire XMPP server (which is what I use), supports user organization by Roster Groups as part of the default setup (just as another "for instance"), as do all the XMPP clients that I listed in the parent post.

What I'm looking for is a recognition like "we know this is part of the core XMPP functionality, we just don't see it as a priority right now", instead of it being "dismissed" as "minor".

All that said, let me be clear that I really do appreciate all the work that has gone into this app. I'd even be willing to pay for it to show my appreciation. That's another source of my frustration - so much about this app is great. It's so close to being "perfect" for my use case, but it's missing a key, core feature that every other client offers.

[Echolon]

@zippydan I understand your frustration, but you know that people here work 100% voluntary. If we don't fix essential functions, a lot more people are frustrated and that puts the whole project to a risk. It is actually a little wonder that mainly Thilo and Friedrich created almost total rewrite of the app in the back-end and now adding essential feature such as group chat, stability and maybe even AV calls. This is the priority. To decide what to focus on we need to set this to minor.

However, roaster groups it is simply not an essential feature yet. But it also hasn't been closed. So, it might come at a time. I think you are putting too much into the label 'minor'. This mostly helps organizing limited resources. Many other minor issues still have been closed.

In the end we need to respect what the developers actually can do and what they are also happy to work on.

I think I can simply ask for your patience. Let's focus on the the next release with testing. Then we will re-check priorities.

[zippydan]

I really do appreciate you.

I'm most frustrated at Apple for removing XMPP support from Messages. It was perfect, simple, streamlined, integrated into the UI, easy to use, and organized exactly according to my needs. And it was all working just fine. Why remove it? Why???

[tmolitor-stud-tu]

@zippydan I think it's time a developer answers this.

It is nice to hear that, for you, Monal is almost complete and usable (only roster groups missing), but for most other people it is not. Why not? Because substantial parts in the muc implementation are still missing. What parts are missing? In Monal 4.9.2: evereything, in the upcoming Monal 5.0: mostly only ui parts, like creating or deleting groups, managing group membership, inviting people to groups, promoting people to admin etc.

But these parts are crucial for any messenger today. It's nice to hear that, for your enterprise usecase, these missing parts don't seem to matter much, but for most other users they do matter a lot, even for some enterprise users. That means: these missing parts are priority issues and will be fixed first, everything else second. The label "minor" means just this: a thing that (for most users) isnt' that important compared to other missing parts in Monal.

However, it does NOT mean we will never fix this(we close issues as wontfix in that cases and don't keep them open), it simply means that we first need to do some other more important stuff.

That said: if you do want to get roster groups in Monal implemented faster, you can always try to get us more developers. Right now we are barely 2 developers working on Monal. Any new developer would help us a lot more than any money could ever do.

Last but not least: I myself need roster groups for a project based on Monal. But even for that a working muc implementation is far more important than roster groups. I won't forget them and I will implement them if the time comes.

[tmolitor-stud-tu]

Selecting option 1 alone would behave like Conversations, whereas selecting option 1 and option 2 together would behave like Xabber.

Conversations does allow sorting by group tags, just tap onto them and it will only display contact having the same tag.

[zippydan]

@tmolitor-stud-tu

Conversations does allow sorting by group tags, just tap onto them and it will only display contact having the same tag.

I'll have to quibble with you on that point. If you read through the rest of the comment that you are quoting (I've just edited it for legibility and coherence), you'll find I go into more detail regarding Conversations' behavior regarding Roster Groups tags, and why it isn't ideal for many use cases.

To be clear on the quibble:
Conversations does not support sorting by group tag.
Conversations does support filtering by one specific group tag.

[tmolitor-stud-tu]

To be clear on the quibble: Conversations does not support sorting by group tag. Conversations does support filtering by one specific group tag. Agreed