Discussion: Matrix Integration

[sunjam]

Specific discussions should take place in the subtickets.

This is a discussion thread for matrix.org integration as mentioned in meta thread #687. Also mentioned in chat meta thread #294 #395. Matrix has been announced as the base platform for the upcoming Librem 5 phone.

[GoetheG]

An interesting idea you have, @sunjam. The college society I volunteer for, tested a lot of different collaboration software. We tested Slack, Mattermost, Riot.im/Matrix, Rocket.Chat, stackfield, Threema Work. From my non computer scientist point of view, it would be the best solution to closely cooperate with Rocket.Chat. I know that Rocket.Chat has not a PHP basis. But I honestly think that this is the best solution from a user experience perspective. Rocket.Chat can be installed on an own server and can also be hosted somewhere. I have NETWAYS in mind.

And for guys like me, who just want to have a good looking and working peace of software, which is easy to update and to install, Rocket.Chat seems to be the way to go. Matrix is not easy to use. It's for professional coders only. What makes Nextcloud really special is it's simplicity. I can easily install Nextcloud within my web hosting service and just use it. Great work you do by the way @nickvergessen. And Matrix is not easy anything. I was shown the installation process on a test server. It's really horrible.

And if we really want people to switch from Skype and Slack and Dropbox, that this open source society should deliver not only a good looking software, but an 'easy to handle' software too.

[natrius]

@GoetheG In my opinion the matrix synapse server is not hard to install. It seems complicated, i understand that, but its not that hard and it just works out of the box afterwards, without much hasseling around. But i have not tried rocket.chat. The usage of matrix itself its easy too - but that should not be point of this discussion.

@Topic I would appreciate the integration of Matrix.

[violoncelloCH]

I would rather go for implementing a matrix compatible api at least for federation. This would open a possibility to connect to matrix instances from within nextcloud talk. Or maybe it could even be possible to implement matrix's client api so existing matrix clients like riot could be used with nextcloud talk

[GoetheG]

When I posted my first comment I didn’t know anything about Nextcloud Talk. Is there maybe anything planned?

I wish there were some easy way to connect Rocket.Chat with Nextcloud. That would make Nextcloud a real collaborative solution. But it seems that Nextcloud is building a completely new “messenger” from scratch.

[nickvergessen]

Well we want everyone, that runs Nextcloud, to be able to also run Talk with as many features as possible. When looking into the federation of chats/calls, we will surely think about all this again and what the best way is to achieve this.

[sunjam]

What @violoncelloCH mentioned is really interesting. I had not thought of Talk being supported by Riot, which is a front end for the backend Matrix Synapse server. There are actually many client front end implementations being used. It would be fascinating if these front ends could be used with Nextcloud Talk.

[entfrickler]

Or if we had a matrix bridge which integrates Nextcloud Talk into the Riot client / Matrix network.

[matthiasbeyer]

As a user, I'd vote against a centralized solution like Slack, Mattermost, Rocket.Chat, Threema.. and would explicitely vote for Matrix, because it is federated.

Just my 2ct.

[sunjam]

Definitely some confusion in that last response. Matterbridge (linked in another request), can interconnect these services via python.

Threema and Slack are proprietary services. Mattermost is FOSS. Rocket.chat and Matrix are both Federated and FOSS. Hope this helps.

On Fri, Aug 9, 2019 at 6:37 AM Matthias Beyer notifications@github.com wrote:

As a user, I'd vote against a centralized solution like Slack, Mattermost, Rocket.Chat, Threema.. and would explicitely vote for Matrix, because it is federated.

Just my 2ct.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nextcloud/spreed/issues/809?email_source=notifications&email_token=AANUKZX2C4BMURFH7OBHNBTQDVXH3A5CNFSM4E4KOWUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD36WB2Y#issuecomment-519921899, or mute the thread https://github.com/notifications/unsubscribe-auth/AANUKZU2G5LIQV4ORRR7SCTQDVXH3ANCNFSM4E4KOWUA .

[gtbuchanan]

I was all for this integration initially, but recently discovered the following privacy issues with Matrix:

https://github.com/matrix-org/synapse/issues/4540 https://github.com/libremonde-org/paper-research-privacy-matrix.org

A new protocol called "The Grid" is being proposed and developed to address these issues:

https://gitlab.com/thegridprotocol/home https://gitlab.com/kamax-io/grid/gridepo

Seems to be led by one dev at the moment, though.

[Xananax]

Is there any official statement answering those points from Matrix?

[gtbuchanan]

@Xananax The research paper claims Matrix did a full review of the notes here and he responded. Though, there's a lot of information to sift through and apparent contention between the two individuals.

[tilosp]

Is there any official statement answering those points from Matrix?

https://matrix.org/blog/2019/06/30/tightening-up-privacy-in-matrix https://matrix.org/blog/2019/09/27/privacy-improvements-in-synapse-1-4-and-riot-1-4

[Xananax]

It seems to me like the second part of the research is either old (written prior to the links @tilosp posted) or not entirely accurate. The response by Matrix in those blog articles seems to commendable to me and addresses a lot of (if not most/all? I didn't do a bullet-by-bullet comparison) the criticisms of the paper. I see this as a constructive attitude, far from the bleaker picture research paper #2 painted.

[gtbuchanan]

I agree though the second blog link from @tilosp was posted only yesterday. Matrix seems to have taken significant strides to address the issues listed in the paper. They also posted an AMA on /r/privacy yesterday. Seems I commented a day too early!

[wrapper]

+1 for this topic, I'd really like my self-hosted communication platforms to be able to integrate with each other

[blaggacao]

Yes, please. Matrix has a wonderful perspective to put all those slacks, telegrams, mattermosts, keybases, IRCs, and - eventually - even Whatsapp into that single box through https://dimension.t2bot.io/ with the right bridges...

My use case: consolidate customer care channels, professionally.

@nickvergessen You might even consider (seriously, and over time) to base your real time communication stack on matrix (giving it some time to mature).

[dan-elevate]

I would also advocate to link TALK with the Matrix ecosystem - would be a great and self empowering combo!!

[johnjaylward]

I too would appreciate matrix integration. Tbh I have not been impressed with the built in Talk. The chat is super basic without even offering a way to add/upload attachments or pictures and the video and voice chats were super spotty while on my own LAN.

I think it would allow the nextcloud team more focus on the "cloud" aspects to offload the Talk plugin in favor of a matrix integration. I think this would line up more with how the email integrations work where nextcloud isn't really the provider but just another client.

[vranki]

It's already possible to embed Element Web as an external application if you find one that allows embedding. SSO and proper integration would be of course appreciated.

[ghost]

Just another comment on how Matrix integration would be very appreciated passing through.

[4jNsY6fCVqZv]

Here's what Matthew from Matrix says about it. An encrypted search could also be implemented. https://github.com/nextcloud/spreed/issues/1437#issuecomment-586468169

[stenwt]

Now, it's possible (though in very early days) to run a Matrix server in the browser, removing the need to run and associate with a homeserver: https://matrix.org/blog/2020/06/02/introducing-p-2-p-matrix

[nickvergessen]

This should be possible now with the Matterbridge which we started shipping in Nextcloud 20. See https://nextcloud.com/blog/bridging-chat-services-in-talk/ for more information and feel free to comment or open new issues if something is still missing.

[benparsons]

We should record here that Matterbridge's relaybot-style bridging is not a complete subsistute for Matrix integration, which would deliver a much more user-friendly bridging solution if integrated with Nextcloud.

[nickvergessen]

Right, seems I misunderstood this then. But at least it lowered the priority a lot then

[plinss]

Hopefully the priority isn't lowered very much. While the Matterbridge solution is something, the UX is not very user friendly.

Also, if the solution here is bridging rather than switching to Matrix protocol entirely (which I would prefer as it provides nice support for federation), double puppetting needs to be part of the solution, something Matterbridge can't (yet?) do.

[sunjam]

Hmm, we should clarify what people in this thread are thinking regarding further integration. If you have any screenshots or explicit examples please illucidate.

The earlier idea mentioned that stuck out to me is using the matrix client front ends on desktop and mobile with Talk.

On Wed, Jan 27, 2021, 7:43 AM Peter Linss notifications@github.com wrote:

Hopefully the priority isn't lowered very much. While the Matterbridge solution is something, the UX is not very user friendly.

Also, if the solution here is bridging rather than switching to Matrix protocol entirely (which I would prefer as it provides nice support for federation), double puppetting needs to be part of the solution, something Matterbridge can't (yet?) do.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nextcloud/spreed/issues/809#issuecomment-768374950, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANUKZWNUMH6ZW6KDTZWX7DS4AYDRANCNFSM4E4KOWUA .

[plinss]

To be specific, what I'm thinking regarding integration is ideally changing Talk from being an XMPP client (and including an XMPP server) to just being a Matrix client. You should simply be able to point Talk at an existing Matrix homeserver, and there could be a separate app that adds a Matrix homeserver to Nextcloud (most likely Synapse) for those that don't have one available.

Personally, I use Matrix as my primary chat service and I bridge it to everything else I need in order to talk to the people I need to talk to. My goal is to be able to stop using all other chat clients and have only one. The last thing I want is to run yet another chat client on all my devices just so a subset of the people I interact with can reach me in yet another walled garden. Furthermore, the Talk experience for text chat is considerably behind what most Matrix clients can do.

Where Talk beats Matrix (so far) is on group video chat. Matrix does 1 on 1 video chat fine, but for larger groups hands off to a (not all that well) integrated Jitsi widget. With the HP signalling server and a Janus gateway, Talk essentially matches what Jitsi can do (once SIP support is online).

Rather than see Talk try to replicate all the things other services can already do, I'd rather see the Talk expertise on video conferencing (and the Spreed/Janus back-end) integrated into Matrix, and be usable with all Matrix clients. Then the Talk app could focus more on adding a seamless integration with Nextcloud, e.g. file sharing, matching Nextcoud users with Matrix Ids, etc. and get the rest, like federation, encryption, rich text, and search, from Matrix where these are all solved problems.

In addition, Matrix already has a much better bridging ecosystem than what Matterbridge offers. By being a Matrix client, Talk would get to take advantage of that without having to re-invent bridging.

Keeping the existing XMPP back-end and having a good (i.e. double-puppetting bridge) to a Matrix homeserver would be acceptable, but it doesn't help having federated chats, and leaves all the rest, like e2e encryption to be solved somewhere down the road.

[sunjam]

In addition, Matrix already has a much better bridging ecosystem than what Matterbridge offers.

Thanks, could you get real nitty gritty on exactly what differentiates the Matrix bridging from the existing Matterbridge. Any screenshots or other details super appreciated. Cheers!

I'd rather see the Talk expertise on video conferencing (and the Spreed/Janus back-end) integrated into Matrix, and be usable with all Matrix clients

Perhaps this would be worth opening a feature request on the Matrix/Synapse end!

[plinss]

There's a good resource on Matrix bridging here: https://matrix.org/bridges/

Matterbridge falls into the "Bridgebot-style" category, which is generally considered to be the worst possible way of bridging systems.

Perhaps this would be worth opening a feature request on the Matrix/Synapse end!

A good solution for group video conferencing on Matrix has been a long-outstanding problem. I'm not up to speed on developments there but from what I've seen the latest is integrating with private Jitsi instances. I recall seeing discussions of whether Synapse itself can be the signalling layer or not (my guess is that it's likely not fast enough).

I'd love to see a feature request on Matrix/Synapse/Element to integrate Spreed/Janus rather than Jitsi, but that would probably be better coming from Spreed developers who are willing to make the effort than me. i expect their efforts would be welcomed.

[Extarys]

There is a lightweight version of the element web client called hydrogen which could be a nice starting point.

[Mannshoch]

Unfortunately, I am not familiar with the technical implementation but in my opinion, users/Nextcloud admins should be able to decide whether they want to store their login credentials from other networks on Nextcloud or be represented by a bot on the other network. At best, you just use Talk for all communication afterwards. It would also be important to be able to easily manage the different networks of the chat partners in the address book. Since the terms and conditions disaster of Whatsapp, I have now installed 5 different messangers an my phone. Various of my contacts are also on the road in several messangers. If I want to make chat groups now, I always have to find out which messanger is the lowest common denominator. Ideally, Talk could do that automatically. I create a group. I invite people and Talk recommends which networks are best to use to create the group/room. For this there should be the possibility to add the user id's of all supported networks directly from a chat to an existing user in the address book.

[a-0-dev]

I just had the same idea (of integrating matrix to spreed/talk) and am extremely pleased there's already a discussion thread on this. As matrix is currently developing a native VoIP (and video?) protocol section, I would suggest waiting for that and then attempt a complete rebuild of Talk/Spreed, reusing the user interface and exchanging the message/videoconf layer completely by matrix.

As far as I know this would remove hosting TURN/STUN servers, which is an advantage in itself - and federation would not be an issue any more, as it currently is with Spreed/Talk. Publishing it as a separate app may make sense to not destroy existing setups though...

[ShadowJonathan]

I'd suggest watching the latest Matrix Live, from this timestamp onwards, for some more context of what they're planning for it to look like.

More info is available at this MSC, which describes the SFU federation they have in mind.

[Thatoo]

I'd love to be able to use Talk or whatever name would be a very well integrated matrix client into nextcloud.

So far, at work, we reduce our collaborative tools to two : Nextcloud (with several apps) and Element (desktop/android) and the only inconvenience we face time to time is the lack of a well integrated matrix client into nextcloud.

We don't want to have one more chat/video system so we don't use Talk (adding one more communicating system is a no go) even though all its integrated feature would answer our needs. We would not switch to Talk and forget Matrix/Element because usage of Matrix is wider.

[mstyp]

I love talk because of how closly integrated it is with files, tasks, and cards. I use it for pretty much all my internal communications. I used mattermost to bridge over a bunch of comms and its so nice to able to quickly

But I also hate talk because no one else uses it. I tried using the element app in nextcloud but since you cant do things like quickly turn comments into cards or spin up chats on a file, it seemed kind of perfunctory.

[Xananax]

While being a big advocate of Matrix and wanting Matrix in Nextcloud myself, I need to also say it doesn't come for free.

I've ran a small Matrix server for about a year or two, and compared with running a Nextcloud server, it's work. I stopped about a year ago, so things may have changed; but I don't think they've changed dramatically. Here are some saillant points:

1. Much bigger hardware requirements. You cannot run Matrix on a small dinky server, whereas you can completely run Nextcloud on a potato. If you intend to connect to the main hub (Matrix.org), your requirements will also grow. They will grow for each public room on another server that any user will join. Which brings me to point 2:
2. Changing requirements: In the 2 years I ran a Matrix server, I migrated it 4 times. In the 5 years I ran Nextcloud, I changed requirements exactly 0 times
3. Instability: Matrix crashes, does weird things requiring you to intervene directly on the Postgres database. And not just on updates either. Things can happen for random reasons. Twice, the system failed so catastrophically that it was impossible to rescue, and I had to scrape all users, all rooms. That is with support from people who make Matrix; they weren't able to help
4. All these issues get compounded if you want to also have your own identity server; if you don't, then you depend on Matrix.org's identity server, which is not the kind of promise Nextcloud gives (owning your data). If you will run your own, then everything I just mentioned gets compounded complexity. I'm also not mentioning integrating Jitsi, or maintaining bridges, etc.

In short: while I can install Nextcloud and forget about it for years, a Matrix server absolutely, resolutely, requires a dedicated system administrator with some time allocated to Matrix. It is not something you just run and can expect to leave running.

To be clear, this is not a diss at Matrix; I love and support what they're doing; I will continue to use it. But a system of this kind of networking complexity isn't going to be immediately robust, not before years of mainstream usage and refinements. That it is so good already is a testament to the dev's talent.

But in retrospect, I do not think it fits as a default chat app for Nextcloud, which is a much more solid offering that people without much experience can hope to install and run. It may be a good contender for a plugin. I want to continue being able to tell my friends to install Nextcloud, that it's easy and there's no good reason not to do it.

[mstyp]

In short: while I can install Nextcloud and forget about it for years, a Matrix server absolutely, resolutely, requires a dedicated system administrator with some time allocated to Matrix. It is not something you just run and can expect to leave running.

For what its worth I've been running the synapse server that comes bundled with yunohost on an old 2010 laptop for about 6 months now, and have not had any issues beyond this facepalm moment which was entirely an issue with me not getting mautrix and not with thes synapse server.

My server does not federate with other servers though, its just a signal / whatsapp / slack / telegram / etc bridge, which I suspect helps helps a lot.

[johnjaylward]

I've had the opposite experience. For me, Nextcloud upgrades foo-bar almost every major version, while Matrix-synapse, the worst that's happened to me has been that I've had to revisit the config file and disable a deprecated option and enable the new one. Matrix has been a much smoother system for me to manage than Nextcloud.

[Xananax]

Yes, without federation, you're ok. With federation things get difficult fast. It increases not linearly, with each server you're federated with, but with:

1. every federated user
2. in every federated room
3. for every local user is in

A single user joining just one single of the big rooms on Matrix.org will already put a small VPS to a healthy testing challenge. A few users joining a few big rooms will melt it.

There are workarounds and ways to deal with that, none of which are accessible to people who don't have in-depth familiarity with using tmux through ssh or controlling a database from a terminal.

I should add also that Matrix working on ways to mitigate, maybe solve, those issues. But we're far from being there last I looked.

[ShadowJonathan]

@Xananax i need to correct you, what you're referring to as "Matrix" is in this case Synapse, which is a matrix server implementation.

I'd like to see how conduit could compare in your experience.

A single user joining just one single of the big rooms on Matrix.org will already put a small VPS to a healthy testing challenge. A few users joining a few big rooms will melt it.

Synapse is notorious at this stage for being poorly scaled to small single-user instances. Conduit might help a ton with this; It successfully runs itself on a 1-2GB VPS, connecting to "Matrix HQ" (the largest matrix room with over 35k members, and a ton of "state" to process), and after a bit of churn, staying up and running with minimal overhead.

for every local user is in

This is misleading, from a federation and resource perspective, a server joins a room, so if 1 or 10 local users have joined a room doesn't matter much, because only then the total size of the room matters for resource purposes.

[motey]

@Xananax

Sorry i can not share your experience. To be honest it is the complete opposite:

I invested a lot of time, effort and money to get my Nextcloud setup snappy, transportable and reliably. Every app update is still a certain thrill :) dont ask about the major upgrades.

Synapse? I just deployed the docker container, adapted the config and it worked since day one. (had some hickups/work with ma1sd/ldap but to be fair that is not part of synapse)

Memory consumption in large setups are an issue with synapse, but it keeps getting better and better.

But whatever, all these "which implementation of another chat system works better for me"-discussions fail the point!

The scope should be on

• Which standard has the highest chance of having a long live?
• Which standard is truly open?
• Which standard has a certain completes?
• Which standard is of the highest quality?
  • Which standard has a large community, reliable maintainers, backers?
  • Which standard is well documented?

And if one judges from a perspective like this, matrix is the winning solution for me.

[Extarys]

FWIW: Synapse is very heavy. Dendrite is an official implementation in Go.

[Mannshoch]

• There are two seperate point we have to consider. Matrix integration as client and Matrix integration as Server. At least I would like to see a Matrix Client Integration. Every user should be able to setup his Account in the Profil and have acces to it with Talk.
• Then the Server/Federation. It would be nice if a Server is possible. Here I would simply like it if I cloud access Talk with a Matrix client.

I would priorize the Client integration. The Talk App could than grow to a unified App for several Protocol and connection. Also the Server/Federation part would be important.

[ptman]

@Extarys Dendrite isn't necessarily any lighter, at least yet: https://matrix.org/blog/2022/03/04/this-week-in-matrix-2022-03-04#dendrite-website

[Xananax]

I don't think personal experiences matter that much. Mine neither. I wasn't offering it as proof that Matrix is less stable than Nextcloud. I do tend to believe it is (I'm often installing VPS for small businesses, startups and small groups, so this isn't just a one-time thing), but I do not care overmuch which is less stable.

Rather, the point I failed to make properly is that adding Matrix to Nextcloud is an immense added complexity and dependency, on top of an already immensely complex system. That alone makes it, in my opinion, not something the Nextcloud team can realistically consider.

Believe me, I do want Matrix in Nextcloud. I am an early adopter of both, and I'm above in this thread, 2 years ago, demanding for it. My position isn't an emotional one. I just think it's too big a dependency.

As for the hardware requirements, that one is I think objectively beyond discussion.

As for the questions of

• Which standard has [...] until
• [...] well documented?

I don't think these are good questions, even if they appear to be, and I do not wish the conversation to go that way.

Firstly because there's no point asking them without context. For example, a rocket is faster than a car, but it's still not really a good recommendation for commuting to work. Matrix may be the "best quality" and "truly open source" (for whatever definition of those you want), but these are not interesting qualities if it means Nextcloud becomes less accessible due to increased hardware requirements, and if the Nextcloud team has less resources on things that matters because it spreads itself thin.

Secondly, you could completely make a case for XMPP here.

I won't, because I don't think it's an interesting path for the discussion to follow; These kind of arguments alwaya boil down to semantics ("but what does 'quality' mean?"). There's no "good" or "truly" anything; it's always context-dependent. Without defining the context, it becomes a contest of who's talking louder. With context definition, it might just be that they have opposite goals.

The context I'm using is that I personally wish for Nexcloud:

1. to remain as stable as it can by not embarking large dependencies
2. to remain lightweight enough to be usable on a $5 VPS (which it objectively is now, I have installs like that)
3. to remain manageable by the Nextcloud team without burdening contributors and employees for replacing a feature that already works

From that point of view, Matrix, no matter how good or open-sourceness, isn't really a good answer to those requirements.

But it might be an answer to your requirements.

The thing is, Nextcloud has a healthy plugin platform, I see no reason why not have this as an optional client, as Mannosh say. That is a smaller scoped project that can be handled independently by volunteers, and lets people free to experiment with Synapse or Dendrite or whatever they like. It changes nothing to the project itself, adds no additional pressure, and I'd bet the large majority of people who care about this feature can install a docker synapse too, so that part isn't the problem.

[Thatoo]

The thing is that we were talking about Matrix integration within Nextcloud and the conversation became, 18h ago, matrix server integration (a debate about synapse, dentrite or conduit)...

Integrate a matrix client as well as Talk is integrated would be enough for most of us. The way Element is integrated now is not satisfying apparently or we would not have this discussion. This nextcloud matrix client would offer to connect to any matrix server. I don't see why, like it has been done for collabora online, there could not be a matrix server app to install for those who want it (maybe a light one such as conduit) and don't have one out of nextcloud. @Xananax you would not be forced to install it if you don't want to.

The solution could even be to integrate within Talk a bridge to matrix but it would have to be easy for users to set (that is my worry about this solution), make it looks like Talk is a matrix client. It would have to look like Talk is a matrix client. So you can keep using talk the same way but not only with other people who have a nextcloud account but also with those who have a matrix account.

[Xananax]

The thread has mostly not been about having a well integrated optional client, as an alternative to the regular Nextcloud chat. From the very first post back in 2018. The idea is to replace talk with Matrix, and have it as a default offering.

Since using Matrix.org's server by default is antithetical to what Nextcloud is about and the entire promise, then, using a Matrix client in Nextcloud by default implies necessarily a Matrix server by default.

This is what I am arguing against. The idea of simply having an alternate client, as a plugin, is something I've advocated for and supported in every one of my comments, including the last lines of the comment you're replying to.

However I'd argue it's a different issue, because to me it falls under "community-maintained plugins", and not "Matrix integration"

[sunjam]

The solution could even be to integrate within Talk a bridge to matrix

Bridging to Matrix from Talk exists: Matterbridge app for Nextcloud, which the team ported to PHP for the release of version 20. Here is the Matterbridge app for Nextcloud github repository where you can learn what is supported and what isn't.

You can also learn about Matterbridge project on their github