Proposal to switch chat platform from Element/Matrix

[dashagurova]

CONTEXT In the conda community meeting last week (meeting notes), many shared frustrations with our current chat platform, Element/Matrix. While it's a great open-source project, it hasn't been delivering the best user experience. We talked about the possibility of switching to a different chat platform. For this to work well, we'd need all parts of the conda ecosystem to be on board with this and migrate together (conda, conda-forge, bioconda).

PROPOSAL We’re proposing to consider Zulip as an alternative chat platform for conda ecosystem. It was evaluated as one of the options in the Conda Communication Channels Update CEP, and it might be a better fit for us, even if it comes with a bit of a learning curve.

[EDIT]: Another option to consider is Discord (which was also evaluated by Dave in the CEP) and might be a nice option given its popularity in the community, easy way to host community meetings, office hours, etc.

Opening the floor for discussion here. Please chime in and share your thoughts!

Possible discussion points:

• Your experiences with Element/Matrix.
• Pros and cons of moving to Zulip.
• Other chat platform suggestions.
• Potential challenges and how we can overcome them.
• Scope of work for the migration.

[bgruening]

My experience with the matrix protocol is good so far. I have no desire to change again. In Germany there is a university network building up with decentralised Matrix servers, so every student/member of those Universities already has an account. Imho the model of decentralised, encrypted messaging fits our open-source community well. I guess its important to distinguish between the Matrix protocol and the various clients. Element might not be the best client for Matrix for some users. If I need to vote: Stick to Matrix and give users the choice to pick a client.

[beckermr]

Any suggestions on a better client?

[bgruening]

A list of clients is here: https://matrix.org/ecosystem/clients/ probably depends on the OS you are on. On Android the element app is not bad IMHO. Don't use element in the browser is the best recommendation I have, its not very good. The element standalone Desktop app is good. For Qt fans there is Nheko and the Gnomies are developing Fractal (its still rough, afaik).

[zklaus]

I have tried quite a few matrix clients (nheko, cinny, thunderbird). I keep coming back to the element desktop app on linux. The main issue with the others tends to be a lack of support for threads, which makes it really hard to catch up with the conda-forge chatroom when coming back to it after some activity for example overnight or vacation.

But yeah, overall happy with matrix, I would not vote to switch.

[travishathaway]

I personally enjoy using matrix and don't see any compelling reasons to switch. Here are my pros:

• You can launch it in a browser (very accessible)
• Supports threaded conversations

It doesn't support topic based threads like Zulip or Discord, but I don't think these channels are used enough to warrant wanting those capabilities.

I could be swayed to switch though. I just want to state for the record that I don't see why when our current chat solution works perfectly fine.

[BrenBarn]

I've been using Matrix for a few years and I have mixed feelings about it. I'm a moderator on a Python room on Matrix so I've seen a bit of the pros and cons. (I haven't used Zulip at all so I don't have any opinion about that.)

The main positive I see to Matrix is that it is an open, decentralized protocol. It has other fine qualities --- its basic chat functions are okay, it can be used across multiple devices --- but I wouldn't say it's noticeably better in terms of functionality than mainstream platforms like Discord.

The negatives don't crop up so often, but in a way because of that can be more frustrating when they do (since they can take you by surprise). Encryption is a big problem; it is still distressingly common to get "unable to decrypt" errors for no apparent reason. This may not be a problem for conda though if we're talking about using it for public rooms.

The decentralized and open nature of Matrix also makes it vulnerable in some ways. Sometimes rooms can undergo "state resets" due to a conflict between different servers' understanding of the room state. This can result in users seemingly being kicked out of a room, power levels being changed, or other unexpected changes to room state.

Another challenge is moderation and protection against spam attacks. Because Matrix is decentralized, anyone can create unlimited accounts or even create their own server, and there's no central authority that can block this; it all falls on room admins (and to a lesser extent on server admins). Because it's an open protocol, it's relatively easy to create malicious bots that will flood rooms with spam.

In contrast, the moderation tools available within the clients are rather limited, and are not adequate for responding to mass spam attacks. There is for instance no simple way to mass-delete messages, or to ban a user from multiple rooms at once. For robust moderation you need a bot like Mjolnir or Draupnir, which then requires its own setup and maintenance.

In addition, there are some questionable aspects at the protocol level from a moderation perspective. Message deletion is handled by sending a "redaction" event. This means that processing N deleted messages requires processing twice as many events as just processing the N messages on their own (because the server/client must process the N messages plus an additional N redaction events). This in turn means that even a "successfully" cleaned-up spam attack can result in a very slow and clogged room, since the room still has chug through processing a bunch of redaction events. (There is a proposal to add a mass-redaction event, but as far as I know it's not completed yet.)

Like I say, these moderation issues don't come up often, but when they do they can be crippling; a drive-by spam attack (possibly by bots) can happen quickly and take considerable time to clean up, and can leave the room in a degraded state even after the cleanup is done. It's just something to keep in mind.

Finally, I have to say I'm less optimistic than I used to be about the future of Matrix, partly because of the (to me) uncomfortable balance that its leaders are trying to strike between openness and corporatism. Although other clients exist, Element is the most full-featured one. Element is made by a for profit-company. This isn't in itself a problem, but there is a large overlap in personnel between that company and the Matrix Foundation that ostensibly controls the protocol itself. Over time, I've gotten the sense that this has in practice made things relatively more difficult for those developing other clients. Although in theory the protocol and clients are separate, in practice there is a rapid co-evolution happening between the protocol and the flagship client (as well as the flagship server implementation, which has now also been effectively transferred to the Element company) that can be hard to keep up with if you're not "on the inside" of that circle.

This doesn't mean Matrix is bad, but it just means that what I see as its greatest advantage (its open nature) may not be quite what it seems at first glance.

The main alternative I see people comparing to Matrix is Discord. From a pure UX perspective, I don't see that Matrix has many advantages over Discord. Discord is unencrypted, but as I mentioned encryption is one of the shakier areas of Matrix, and in any case encryption isn't of much concern if our rooms are public. Obviously, Discord is a proprietary service controlled by a third party, which could raise concerns about conda's ability to autonomously run its own chat service via Discord. But the same is true of Matrix unless conda is running its own Matrix server (which, judging by the username domains of users in the room, I don't think it currently is). Matrix can provide benefits in terms of increased control, but my impression is most of those benefits are hard to get without hosting your own server (which I think conda isn't currently doing, although I could be wrong). In effect, using Matrix may result in dependence on the Element company just as much as using Discord results in dependence on Discord Inc.

That was long, but hopefully there were some nuggets of useful information in there. :-) Unfortunately I'm not aware of any chat platform that's clearly the best; I just see different ones with different tradeoffs. So far I've been okay with Matrix as the chat platform for conda. I don't see any immediate reason to switch, but if we were already using something like Discord I'm not sure I'd see any reason to switch to Matrix either. I'm curious what problems people had with Matrix (I didn't see any specifics in the meeting minutes). If we want to switch because of perceived problems with Matrix, I guess I'd say we want to make sure whatever we switch to actually fixes those problems, and think about what different problems it might give us instead. :-)

[beckermr]

The main frustrations are encryption errors and threads being kind of lame.

[BrenBarn]

The main frustrations are encryption errors and threads being kind of lame.

Okay. I agree encryption is a problem and as far as I can tell there's no real solution in sight.

I agree threads are lame but I sort of think chat threads are lame as a concept. :-) That is, I personally don't think real-time chat-type communication is a good fit for threads. I'd rather have threaded discussions in a more "persistent" format like the Discourse forum.

[beckermr]

I agree threads are lame but I sort of think chat threads are lame as a concept. :-) That is, I personally don't think real-time chat-type communication is a good fit for threads. I'd rather have threaded discussions in a more "persistent" format like the Discourse forum.

Yeah you are correct about that. At one point conda-forge had an effort to use github discussions but that died out. conda-forge folks really like one chat room.

[jezdez]

I'd be on board with moving to Zulip, maybe to a shared server for both the conda and conda-forge communities? I found the case study related to Rust's Zulip use compelling to our diverse community.

I'm less interested to move to Discord given that it requires a higher level of engagement from participants in my experience, that quickly becomes unmanageable.

[jaimergp]

My crux with Matrix and Element is the thread support and the lack of moderation tools. Trying to provide users with answers to their problems becomes unmanageable. There's no concept of "solved", the UI for the last threads you replied to isn't good enough, and I constantly get lost trying to find a conversation that might have some answers. Discourse's UX is superior here and I would like to have something more lightweight for those users that do not wish to engage in a full forum (feels more "serious", requires more "effort", etc). Chat platforms seem more casual and less intimidating for beginners too, for some reason.

In Zulip, if you want to start a conversation, you need to put a title to it, and you get a thread. It's threads first, there are no other options. Even if you hijack a different thread with a different topic, moderators can splice the thread and move it to a new one, with very little effort. I can also mark a conversation as solved when the user has fixed their problem or obtained the required help.

There are private rooms too, for steering council and core.

So I think it has everything Element provides, but with superior UX. And the search actually works :D

[bgruening]

Are we not mixing up here two different things? A chat platform and a help forum? At least in our community, we use a chat platform for organisational stuff, loose discussions and drive-by chats - pointing people to documentation and discourse etc ... What is the purpose of https://conda.discourse.group if we don't nudge people to this site and make Q and A more accessible and findable?

[BrenBarn]

I agree that a forum is more useful as a place for general user help. A big problem with chat-based systems is they're usually not findable via web search engines, which makes them much less useful as a general knowledge base. With regard to "casualness", from what I've seen, the main thing that makes people feel like something is high-effort is that they have to create an account rather than being able to post as a guest. Not sure whether Zulip or Discourse support guest postings. There is also a bit of a tension though between making things low-effort for users and making moderation easy, because the lower the bar is to participate, the more potential for a lot of low-quality participation.

I guess this is kind of broadening the discussion but I feel like chat is better for synchronous or real-time discussion, and maybe for internal dev discussion where the traffic is relatively low and there's a small set of users who all more or less have an idea what one another are talking about. For cases where the discussion is likely to stretch over multiple days and/or involve a broad mix of users contributing over time something like a forum is better. So sort of like I said above, my perspective is basically that if we need threads for something we should just be directing people to the Discourse forum for that instead. :-) The forum is pretty nice but I notice it doesn't get much traffic right now and sometimes questions sit with no reply for a good while.

[jaimergp]

Just came across this thread in the jupyter org: https://github.com/jupyter/governance/issues/182. Many of the same comments.

[jezdez]

The plot thickens!