Should we run a Mattermost server?

[zerebubuth]

Mattermost is an open-source clone of Slack, an IRC-like messaging platform for the web and mobile. Unlike IRC, Mattermost is centrally-managed, and stores logs for full-text search and "scrollback". It also has mobile integrations.

This ticket is for discussing whether it's a worthwhile service for OSMF to be running, and investigating the feasibility of running it.

[gravitystorm]

I think it's something worth using, but there are various options as to how we could make this work.

In the short term, is there any white-label service available? For example, can we point chat?.osm.org at someone else's servers and pay them $5 per month or similar?

[zerebubuth]

Enterprise pricing from Mattermost is $20 / user / year. For the ~90 people in the weeklyOSM channel that would work out at $150 / month.

[mtmail]

$50/month flat for 1000 users because OSM would probably qualify as charity https://about.mattermost.com/mattermost-mondays/

Been corrected: $50/YEAR

[fredao]

$50/year - just bought such a licence ;-)

[zerebubuth]

@fredao - for weekyOSM or something else?

[iandees]

As a reminder, you're all welcome to use the OSM US Slack team. It's free, we have scrollback, search, ~90 active users, and room for ~140+ more active users.

[lbarrosop]

@gravitystorm about your question I think is answer here: https://www.mattermost.org/what-slack-might-learn-from-its-open-source-alternative/ Private Cloud Deployment

Mattermost is built for private cloud deployment, from supporting mobile app push notifications from behind your firewall, to running as a single Linux binary with MySQL or Postgres on a host of popular platforms, dozens of open source self-hosted integrations, to fostering a vibrant community of containers and orchestration integrations from Docker and Kubernetes to Cloud Foundry and Puppet.

Over two years ago, Slack announced a self-hosted offering but reversed course in 2015.

Self-hosting provides some enormous advantages, such as privacy, added security, and resilience to outages and DDOS attempts that can bring down public cloud services.

Most importantly, it lets you control your own data.

[zerebubuth]

@iandees - Thanks. Part of the original driver for Mattermost was that some people are uncomfortable using Slack, unfortunately.

[tomhughes]

Resilience to outages and DDOS is a two edged sword - a large provider is more likely to be attacked that a self hosted instance but is also far more likely to have good defences in place.

[fredao]

@iandees ... the weeklyOSM team uses Slack for the communication beside the communication via our own tool OSMBC [1] .

Slack is not well seen by several people because it doesn't fit to an Open Source project like OSM. As I see is HOT now using Slack as well. weeklyOSM has integrated about 90 people into Slack - some others don't accept it at all.

weeklyOSM has the strong wish to move from Slack to Mattermost [2].

Wambacher [3] one of our team-mates is examining the technical part of Mattermost at the moment. He gave me a short view into Mattermost as a user and group-admin. Afaics it is really compatible to slack. Even the menus have the same names. So, people who move from Slack to Mattermost will not have any problem to use it.

[iandees]

Ok. My opinion is that OSM should use whatever communication tools/platforms that help OSM the most, regardless of its open source status. For OSM US at least, Slack has worked very well.

On a related tangent, I want to caution against adding yet another communication platform for the OSM community. Users already have a hard enough time figuring out how to interact with our community, and adding a Mattermost server will make the situation worse. If Mattermost ends up being useful and successful, perhaps we could phase out pointing people to irc.osm.org?

[tomhughes]

The problem with all these new fangled platforms is they want their own clients and won't plugin to the instant messaging client I already run to talk to every other service...

[zerebubuth]

Users already have a hard enough time figuring out how to interact with our community, and adding a Mattermost server will make the situation worse. If Mattermost ends up being useful and successful, perhaps we could phase out pointing people to irc.osm.org?

Or vice-versa?

I realise that the too-many-channels problem is completely off-topic here, but it would be useful if that conversation were being had somewhere. Does anyone know where it might be happening so that we can point to it?

[fredao]

@iandees

Ok. My opinion is that OSM should use whatever communication tools/platforms that help OSM the most, regardless of its open source status. For OSM US at least, Slack has worked very well.

Well, my opinion is not important, I have to see what efficient contributer tell me. If they refuse, it makes the communication more difficult. ... and btw they have great arguments! ;-)

On a related tangent, I want to caution against adding yet another communication platform for the OSM community.

I agree with you, that we have to many platforms. I think nobody knows it better than the weeklyOSM-team, because we try to scan them every week - and we need a spreadsheet to match the editors and the platforms. With the new channels on telegram the information gap increased.

BUT I am not talking about the community! I talk about - in our case an existing group who wants to switch from Salck to Mattermost. AND if you know Slack you can see, that the communication changed and became more efficient. For some organisations in OSM it would be extremly helpful. weeklyOSM runs two slack teams. And we could integrate significant more people if we could offer a FOSS solution. BTW - switching from Slack to Mattermost is no problem at all for the user or the front-end admin, because even the menus are the same.

Users already have a hard enough time figuring out how to interact with our community, and adding a Mattermost server will make the situation worse.

Once again I don't speak nore about a newcomer, neither about the community itself.

[fredao]

@iandees

As a reminder, you're all welcome to use the OSM US Slack team. It's free, we have scrollback, search, ~90 active users, and room for ~140+ more active users.

You know that creating a slack team is less complicated than subscribe to Google-mail. Everybody can do that in one minute. So, that is absolutely not the problem.

[iandees]

@fredao

You know that creating a slack team is less complicated than subscribe to Google-mail. Everybody can do that in one minute. So, that is absolutely not the problem.

Yes, but getting the paid version of Slack for free (because OSM US is a non-profit and has gone through the process of getting it set up with Slack) is not something everybody can do in one minute. I know that some people dislike Slack because of the limitations imposed on the free version, so I mentioned this to say that we could help solve those problems.

[fredao]

@iandees It is not at all a question of the limiation of the free version. We exceeded the limits long time ago in both teams and we could live with the limitation.

Again weeklyOSM has to respect the arguments of contributers who don't join slack to be consistent. And we must give an answer. Lbarrosop posted above some arguments and for those who refuse it is as well a question of a their self-concept: working for the best free map and do it - if ever possible - with FOSS tools.

[iandees]

I'm not trying to convince anyone to switch to Slack. I'm just offering options. Let's get back on topic 😄 .

[fredao]

And we could not convince, so we had to act - that is our topic ;-)

[PedaB]

While I think IRC is good enough, there are also quite many users who want to have these modern Web-chats with all those fancy features. This can be seen just how many OSM-related Slack channels there are (OSM US, HOT, Weekly, ...). So we have to admit that it's a service users want.

@tomhughes Yes, Slack features the possibility of voice chat for individuals (free edition) and for groups (paid edition) and I guess Mattermost does too (I didn't check). Though as a member of the weekly team I can tell you that we are (or I am) not interested in voice chat, so we could deactivate that part. We just want a text chat with history function and image uploading,.. So if that makes a difference in the decision: Mattermost without voice chat is ok :-) And while I'm at it: Slack (and therefore Mattermost?) have a XMPP and an IRC gateway that can be activated for the "old-fashioned" like me ;-)

I'm one of the guys @fredao talked about who prefer Mattermost over Slack. Actually I'd be satisfied with IRC, too, but that does not seem to be an option for those used to Web2.0 and all that. I always prefer OpenData and OpenSource and I was one of the forces behind the FOSS policy (http://wiki.osmfoundation.org/wiki/FOSS_Policy). While this FOSS policy does not apply to WeeklyOSM, I still recognize its values. I.e. iff we need a Webchat we should use Mattermost and abondon Slack. That's something important to me personally and if OWG could help with that would be great!

[tomhughes]

To my mind we run way too many auxiliary services already and we should really try and avoid adding more where we can. We often wind up picking the wrong horse and getting stuck with it (see OSQA which was entirely my fault and which I bitterly regret) and I'd really love to find a way for use to use best of breed solutions run by other people where possible and let us stick to the things that make us unique.

That ridiculous FOSS policy is basically just empty posturing by the board because it's what a politician would call an "unfunded mandate" in that it's just words with nothing to make it happen - the services are run by volunteers who are going to do what they think is best rather than what some policy written in an ivory tower says they should do.

It even comes complete with a farcical attempt to justify the OSMF using google for email by saying it would have been too much trouble to setup and run a mail server, which ignores the fact that we were already running one for openstreetmap.org, that could easily have run osmfoundation.org as well, when the foundation started using google. The honest reason would be simply that google was convenient and "cloud" was hip for the people that made the decision.

Look I love open, but mostly in a practical and pragmatic way rather than in some kind of hard core "making a political statement" way so sometimes using a hosted service is the right way to go from my point of view. Ideally a service that uses open software and contributes to and engages with the community sure, but I just have a problem with doing things for purely idealogical reasons.

[pnorman]

Given the fact this would be a tertiary service that doesn't need to hold private data, isn't OSM specific, and there is an established provider with low pricing, it doesn't sound like something operations should run.

The FOSS policy is not relevant here as it doesn't mandate anything and people are free to set up something like Slack independent of the OSMF, but I would also prefer something open over Slack. But there isn't an official OSM slack, nor does the foundation use Slack to my knowledge, so it's not really in the picture.

With using the hosted solution for mattersmost, I question if the demand is really there. All the people I've heard talking about it are from osmweekly who want to use it for their work on osmweekly, and I haven't seen people asking for OSMF reasons or a broader community demand. Of course, it's a do-ocracy, so if someone wants to set it up, they could, I just don't see an ops involvement.

[simonpoole]

Well given that I've raised the question of alternatives to slack multiple times and have researched things a bit, my two cents:

• the main issue with slack is not cost or no cost, it is requiring our community to sign up with yet another service trying to build a business on siphoning off data from its users, regardless of if we personally care about it or not, we shouldn't be making participating in discussions dependent on our community agreeing to it,
• after some research there are essentially two avenues from a product pov that could work for us: matrix.org (distributed "similar to IRC" service with a number of nice clients including mobile, but early days in development), and mattermost. The later being closest to slack, so likely with more acceptance.
• the free version of matermost, is likely not really suitable from certain capabilities that we would likely want, so self hosting is probably out of the question in any case

@pnorman well I would have some hope that we could slightly reduce fragmentation if we offer a klickibunti IRC variant,

[fnorf]

Have you considered matrix.org? It's a standard rather than another proprietary program, with a multitude of clients, servers, bridges, and and and. It is really powerful.

edit: Sorry, did not not reload before posting so I missed @simonpoole suggesting the same. The flagship client is https://riot.im/ which from what I know about Slack is pretty much that.

[simonpoole]

@fnorf actually my "compromise" position would be to do away with the Web IRC interface and replace that with matrix. But realistically that is unlikely to win back the slack enthusiasts (and if we run our own instance add to the ops load).

[tomhughes]

Can matrix connect to IRC then? and WTF is "klickibunti" when it's at home?

[Nakaner]

tomhuges wrote:

Can matrix connect to IRC then? and WTF is "klickibunti" when it's at home?

"klickibunti" is a German word (adjective) used by nerds when talking about software for the most stupid users.

"Klick" is the German verb for "to click".

"bunti" is a derived version of "bunt" (colourful). You can also see "bunti" as part of "Ubuntu" which aims to be more newbie friendly.

"klickibunti" is not a friendly word.

[simonpoole]

There's an IRC gateway that seems to work reasonably well, and their room concept maps well to IRC channels (see https://riot.im/app/#/room/#matrix:matrix.org for a web client).

@nakaner is giving "klickibunti" a very negative twist, in general it just means a UI with lots of visual colorful dressing :-), with a slight negative undertone.

[Nakaner]

Hi,

this is my personal opinion, not an "official" opinion of WeeklyOSM/Wochennotiz. (Note that I am currently not an active member of WeeklyOSM/Wochennotiz due to my master thesis and an event I am currently organising)

We talked a lot inside the team about a Slack replacement. I was and still am against Slack because it is not only proprietary but also an US company (therefore there is the great privacy problem and foreigners have no rights).

I suggested to move to Jabber server (it's a little bit better than IRC but IRC would serve the job as well) and offered to set up one on my private machine (with multi user chat) but not before end of January 2017 because I am busy with my master thesis. I already run an ejabberd instance but it is rather old and due to a lack of some up-to-date encryption features not able to connect to popular hosts like jabber.ccc.de. In about six weeks my current server will be moved and set up form cratch (new OS version and therefore new ejabberd version). That's what I call "I offer to set up a Jabber service for weekly".

About two weeks ago, Manfred had some arguments which convinced my for two days until I thought about it again and dropped his arguments. To summarize the arguments:

Pro Jabber, against Mattermost:

• Jabber servers are easier to set up. It is not that new and more often used. The documentation is good. I can confirm this because I set up ejabberd. It was easy.
• ejabberd will generate less load on the system than Mattermost.
• Less features will hopefully come with less security issues.
• ejabberd seems to run with almost no maintenance. That's great, isn't it?
• Mattermost has a web GUI. It will make it more accessible for potential future team members who are no nerds. We would hope to have some more members in languages we currently cannot support.
• Mattermost has a nice file sharing feature. Post a file, e.g. an image, to a group and everything is ok.
• Mattermost has logging but there are logging solutions to do that on an Jabber server, too (or you just write a script which does the job).
• Setup seems to be difficult. (experiences by wambacher et. al.)
• I don't know how much Mattermost maintenance will be necessary per month.
• Mattermost is product of a single company but there are at least two Jabber servers. If Mattermost goes bankrupt, the system might die. If the company behind ejabberd shuts down, people will switch to another server or the pain is that large that still some people with Erlang knowledge will care for ejabberd.

I think that disadvantages of Mattermost are larger than the advantages.

simonpoole wrote:

the free version of matermost, is likely not really suitable from certain capabilities that we would likely want, so self hosting is probably out of the question in any case

If self-hosting is not possible, we should no longer think about using it. I think that a software which claims itself to be free and open but cannot hosted by our own is not "free and open". (Not being able to host by ourself does not mean that we lack the staff to care about it.)

Best regards

Michael

[pnorman]

The majority of opinions here seem to be looking at what would work best for WeeklyOSM, not what would work best for the whole OSM community or the OSMF. I'm not sure what the best way to gather the needs of the latter is, but this ticket isn't working.

[simonpoole]

@Nakaner I might have been incorrect wrt self hosting given that it does seem that they offer a discounted rate for the enterprise version that we could likely afford, still self hosting would add additional load on ops.

[PedaB]

@pnorman Perhaps the majority of opinions are expressed by members of WeeklyOSM, but I didn't have the impression that this discussion is about what would work best for WeeklyOSM but why hosting Mattermost by OWG/OSMF might make sense or why it does not.

I find it kind of concerning that there are more and more groups switching to one of those "klickibunti" platforms. Not only Slack, but e.g. also Telegram-groups and what not. And as we have official IRC channels and still users are switching to Slack, there seems to be a need for such kind of chat systems...

Anyways, as this issue seems to get featured in the upcoming WeeklyOSM, perhaps that part of the OSM community which reads Weekly will participate here soon :-)

[zerebubuth]

I'm not sure what the best way to gather the needs of the [whole OSM community or the OSMF] is, but this ticket isn't working.

I agree. I think this is a much wider discussion than whether it's possible for us to host and run a particular piece of code. We need to understand what "official" communication channels are necessary and which of those currently available are unnecessary.

I think we should close this ticket in favour of a discussion elsewhere (but where :scream: ?)

[mikelmaron]

That ridiculous FOSS policy

off topic, but I need to respond since @PedaB and myself did spend time on this. @tomhughes there were important reasons for putting together this policy, even if it is only a small thing. if you had some ideas to make it better, would have been well received back when we were discussing it, rather than holding and finally exposing your bitterness in a GitHub comment months later.

[nicolasmaia]

I support Matrix, as others have suggested in this thread. Distributed, secure, libre and interoperable with other protocols.

[RobJN]

@pnorman wrote:

But there isn't an official OSM slack, nor does the foundation use Slack to my knowledge.

Not right. There is an OSMF slack channel that is used by the SotM team and the Belgium community. There was also a slack group set up for the recent OSM awards selection committee.

[RobJN]

Btw I did a load of research on all the available tools for a different discussion but it applies here too. There are numerous slack-like options:

https://docs.google.com/document/d/11Z9vCilV96Tah5ruGFTIaSGxojEC9pvHo48Sc4vw5_Y/edit

See http://rambox.pro a .

[althio]

@fredao If you want to try Mattermost, hosted by a FOSS organisation, have a look at [French] https://framateam.org/ ... I am not sure the translation/integration is good for other languages though.

@zerebubuth I understand the additional burden and I can understand OWG is not using resources for this.

@pnorman @Nakaner and all Several communities are using, trying or considering Slack (SotM-WG+OSM.be, OSM.us, HOT with IRC hooks) or Mattermost (OSM.fr) or Telegram (quite popular in South America) and even Facebook for messaging... IRC and Jabber just do not fly for everyone. A lot of people want web GUI and phone apps, not clients. I never tried matrix/riot, looks promising.

@mtmail https://github.com/openstreetmap/operations/issues/128#issuecomment-264198180

$50/year for up to 1000 end users Mattermost: Not-for-Profit License Request

Board members? Want to spend 50$ for hosting Mattermost?

[fredao]

@althio thanks for the info with https://framateam.org/ - sounds great to me - weeklyOSM will go in contact with them still today ;-) ... and btw ... +1 to you answer @zerebubuth

[zerebubuth]

@althio

I understand the additional burden and I can understand OWG is not using resources for this.

Sorry! That wasn't what I was trying to say. Running software is (relatively) easy, and OWG are happy to run things when they're useful to the community.

What I was trying to say was that on a non-technical level, adding new communications channels is not necessarily a good thing. Each new channel needs to be documented and advertised so that new users can find it, and maintained by a community of users. As @Nakaner posted to the mailing list, we already have a lot of dead or semi-dead channels.

I think having too many channels makes it more difficult for users, particularly new users, to get the help or conversation that they're looking for.

The reason I don't think it's a useful conversation to have here is that it's non-technical. There are many different pieces of software which implement the similar styles of communication, so we should first choose which styles of communication are most appropriate. For example, it's worth discussing Matrix and Mattermost only after there has been a discussion about whether real-time chat is an appropriate "official" communication channel.

My personal opinion is that we should aim to have only two or three "official" communications channels, which might include:

• Real-time chat (e.g: IRC, Matrix, Mattermost)
• Ticket-like / Q&A (e.g: OSQA, Discourse, Github issues)
• Long-form prose (e.g: Forums, Mailing lists, Diary entries, Private user messaging, Wiki)
• Short-form prose (e.g: Twitter, Diaspora*)

Within those categories, there are styles which are "channel-centred" such as IRC, Forums, Mailing lists, and others which are "user-centred" such as Diary entries, User messaging, Twitter, and others which are "content-centred", such as the Wiki and OSQA. Different styles will have different advantages and disadvantages for the people using them, which might make them more or less appropriate for use in OSM. Once we know which styles we want to support officially, then we can evaluate different pieces of software against them.

[fnorf]

By the way, you can use your matrix.org client right now to join the OFTC IRC channels. See https://medium.com/@RiotChat/new-irc-integrations-oftc-and-snoonet-b88883a58303#.qvuqksr9l for some more details from September. For example #oftc#osm:matrix.org. Once someone joined, channel history will be available for others I believe.

This might reduce friction.

[gravitystorm]

Board members? Want to spend 50$ for hosting Mattermost?

I feel I should point out that OWG has a budget, so this topic doesn't require any involvement from the OSMF Board. Particularly because the budget is a very small part of our decision.

[althio]

@zerebubuth Thanks for clarification, I misunderstood some of your goals and other reactions then. Your line of thoughts looks great to me.

[karussell]

Sorry for my 2 cents here but I like the summary of @zerebubuth :) and would point out that discourse has more potential. So hopefully this is not misunderstood as an advertising of discourse :) ... we had really nice experience with it over the last >15 months within the GraphHopper community and here are advantages that I see:

• it is real time. So everything you post via the browser will be there in real time. So you can 'chat'.
• If one topic is too busy you can easily mute it or just do not subscribe to a category or subscribe partly (no notifications will be sent it is just marked as unread)
• Everything that you do not saw online will be sent to you delayed via email (if you like). So this is even a lot better then with github where you get an email even if you already saw it online and a single email for every comment or edit ... producing too much noise.
• For every subproject you can create a new category and subscribe to them or not
• You can use it like a mailing list and create new topics via email
• you use kind of markdown syntax I think
• you can post global announcements, polls, other plug ins
• you can easily search for something
• it is open source and developed by the best (stackoverflow people ;))
• you can migrate away from existing mailing lists or forums (we didn't try this at the end as it looked like too much work for us)
• installation is fast and simple due to the docker image
• maintenance is very simple (often just a click to upgrade from within the web UI), or just use the hosted solution where you can then migrate to a self-hosted system
• There are big communities like ElasticSearch that use it successfully.

What I personally like best (and what I dislike about chat) is that it is asynchronous most of the time, but when you like or need it, you just open the site in your browser and get real time updates. And people that just want to use email can keep using it :)

So this could be used as a replacement for the forum & mailing lists and even act as a slack/mattermost alternative. BTW here is our discussion about a 'chat' alternative with some disadvantages pointed out.

[natrius]

News on this one?

Asking, because i'm using synapse (the matrix homeserver) on my own and installed riot on my devices (android, linux, windows) and i am very pleased.

Spain seems to be active in there with a room with 370 members and there are some other channels to, created on the reference matrix.org server. Germany with just about 14.

Also, there could be community set up "OpenStreetMap" with a description where all various rooms could be collected.

[grinapo]

In my decision tree it is a very high priority point that the chosen method ought to be open, platform independent, desktop and mobile friendly and possibly supported by multiple upstream servers or multiple vendors. (The good thing about IRC was that there were plenty of servers to pick from and they were - mostly - interoperable. Most suggestions here mean to pick a vendor specific protocol with limited amount of clients with serious limitations on some platforms.)

The other way around (which has been the habit for decades in IRCland) to have bots interconnecting the networks, like mattermost - irc - xmpp (or -matrix with seems to be in development). But that only works for real-time channel-based chats.

(I kind of dislike stuff like discourse which lack interoperability, and hides plenty of information behind a convoluted ("advanced and responsible") user interface. It is supposed to be a "ticket-based" approach but it's a web forum without useful hierarchical structure. For Q&A the best structure seems to be the stackexchange-clones (askbot, osqa and others) with a good search.)

[natrius]

Matrix = protocol, Synapse = one server implementation, riot.im = one client (most popular at this moment). So, the following conclusion is a bit a mix about all of the three i mentioned above:

• Its open (Synapse and riot.im)
• platform independent (riot available everywhere, 4 more clients available)
• multiple servers already there. I have even my own, federated with the rest. You could pick mine. Good thing :) (even other server implementations for matrix are available. Synapse is just the main right now)
• no limitation as far as i know
• synapse has bots (its called bridges) right now alread implemented direclty for (i checked) IRC, Gitter and Slack. I know there are much more available like GitHub (i'm in one channel already using this), RocketChat, Telegram, Whatsapp, XMPP.

I don't know if the IRC also works when not connected, but i think it does work.

Again, it depends on what is needed. If there are needs for pinned-posts or something like this, synapse might not be the server to go. (not yet at least). With riot.im it is possible to search a channel, but to be honest, its not good right now. So, Q&A, it would just be a "Just ask again and don't bother to search". shrug

[simonpoole]

@natrius & @grinapo It is fairly clear that if we did want to do something the obvious way to go now, two years later, would be with a matrix home server (given that mattermost is not quite as open as it seems). But the point is somebody (well actually best more than one person) actually has to commit to do the work of providing and running the service in a sustainable and secure fashion. Arranging for HW is something like 1% of the problem.

[amandasaurus]

I don't know if the IRC also works when not connected, but i think it does work.

AFAIK IRC still doesn't work when you're not connected, you cannot see things that were said before.

somebody (well actually best more than one person) actually has to commit to do the work of providing and running the service in a sustainable and secure fashion

Here here. I wanted there to be an OpenStreetMap Mastodon/ActivityPub server (in English), so I just signed up for a hosted plan, and hence we have https://en.osm.town. I don't know anything about Matrix/Riot/chatty things, but why not just do that? Domains can be eventually transfered to OSM if needed. But usage is all that's needed.

[natrius]

I can at least help setting up Riot as i am writing the (somewhat official) guide for setting it up right now and improving it as far as possible.