Conda Communication Channels Update

[tnabtaf]

Hi All,

I wasn't really sure where to put this, but I think it might become a good CEP candidate. We'll see where discussion goes.

---

Abstract

Vibrant Open Source ecosystems have places for the community to gather and engage online. Conda already has several places for doing this. This proposal argues that we might have too many such places and that we would benefit from consolidating and updating our channels for community communication.

Motivation

Conda currently has at least 4 conda-centric channels where people can ask questions, post answers, and publish news. In addition there are several nearby channels that aren't about conda per se, but are about parts of the conda ecosystem. Finally, there are a number of places around the web where people often post conda related content.

The central goal of this proposal is to reduce the number of "obvious" places where people think to post and look for questions and answers. These channels will eventually be promoted through things like conda.org and @condaproject on Twitter, for example.

There are many potential benefits to consolidating and updating:

1. There will be fewer places to search for previous questions and answers, making previous discussions easier to find.
2. Fewer communication channels mean a broader audience on each of the remaining channels.
3. Having fewer, but more obvious places to post queries may result in people who currently post to StackExchange or Reddit (for example), posting more of their queries in conda channels instead.
4. Having fewer channels increases the effectiveness of community building. It's harder to form a coherent community when there aren't obvious places to ask questions and search for information.
5. Reviewing and updating our channel platforms may reduce barriers to participating.

Communication channel consolidation is also an ideal time for us to define what each of our channels are for and to rethink our communication goals.

Specification

We propose:

1. Consolidating around a single "chat" platform for real-time discussions.
2. Consolidating around a single "forum" platform for long-form question and answer threads.
3. Creating a "conda announcements" mailing list.

And, a little more detail:

• Actual issues and feature requests would still end up in the various source repositories. Discussion about issues and feature requests may first happen in the chat or forum channels, before they show up in GitHub.
• We are not proposing (initially) that conda-forge, Bioconda and others migrate their specific channels to the consolidated conda channels. However, the hope is that if these channels are successful, then associated communities might be interested in doing so in the future. We would always welcome this.
• The announcements mailing list would be broadcast only. It would not be a forum for discussion, but rather a place for community news, open to use by all projects in the conda ecosystem
• All of this can be seen as a precursor to, and complementary to, a conda community hub site at conda.org. We hope to have that up later in 2022.

Alternatives

If we were to consolidate, what platforms would we consolidate on?

But, first…

A word about NumFOCUS

The conda Organization has applied for Fiscal Sponsorship from NumFOCUS, and this application is currently under review. If the application is accepted, then can NumFOCUS offer us any options as a sponsored project? According to NumFOCUS:

• They offer free Zoom Accounts
• Discounted Slack Pro Accounts. This is likely the US$1 to US$1.15 per person rate that non-profits receive.

That discounted Slack rate is included in the discussion below.

Chat platform

Conda currently has chat rooms in both Gitter and Slack. What are the pluses and minuses of these options, and what other options are there? Chat platforms tend to be used for a mix of real time and asynchronous communication. Individual posts tend to be shorter in chat than they do in forum platforms.

Common challenges with chat platforms include:

• Searchability
  • How easy is it to search content within the platform?
  • How easy is it for search engines to search content within the platform?
• Linkability
  • How easy is it to share links to particular posts/discussion?
• Threading
  • How much support is there to support threaded conversations, and to avoid overlapping parallel conversations in the same room?

Gitter

Gitter is a widely used platform and it is one of the two chat platforms currently used by conda.

Features and foibles:

• It is one of our current platforms so people understand how to use it.
• Free!
• Searchability does not exist in the Gitter Web view.
  • Search in the Gitter Desktop app is limited to the current channel and to the names of other channels.
• Gitter does not have mobile apps.
• Requires a login on Twitter, Github or GitLab to contribute, but anyone can read..
• Supports threading, but almost no one uses it.
• NOTE: The owners of the Gitter platform plan to deprecate it in favor of Element/Matrix (see below)

Slack

Slack is also a widely used platform that is one of the two chat platforms currently used by conda. It has a nicer user interface than Gitter.

• It is one of our current platforms so people understand how to use it.
• Free version is available - it's what we use now
  • But Slack is changing its free policy from "you can search the last 10,000 posts" to "you can search the last 3 month's posts."
  • Paid non-profit version costs $1 to $1.15 per User per month, which is not sustainable with our numbers.
• Search platforms like Google can't index the content.
• Requires a login to read it.
• Supports threading; has mixed adoption.
• Good search and community management features.
• Limited moderation features.
• Barrier of entry for new community members because of having to use invite links.

Element/Matrix

The same people who own Gitter (and plan to deprecate it) also own Element. Element is a front-end to Matrix-compliant message platforms. Matrix is a protocol for open communication, maintained by the non-profit Matrix.org foundation. Gitter is compatible with Element/Matrix, with one exception: You can't see private groups from Gitter in Element.

• Element feels a lot like Slack, so it is easy for most people to learn
• It's still another platform that many people will not have prior experience with
• It’s open source and free
• It's basically a better Gitter
• Supports threading
• Matrix-based rooms are published on the web but they do not appear to be indexed by Google.
  • Google can find the archive, but doesn't seem to search the archive.
• Element has a web-based interface, and an app-based interface.
  • You can view rooms when you aren't logged in
  • But the web interface does not appear to support in-room search, unless you are logged in.
  • And this path is not indexed by Google either.

Zulip

Zulip walks a middle ground between chat and forum platforms. Every conversation on it is threaded and has a title, but the feel of each thread is like a chat. It is possible that Zulip could be the single Chat/Forum platform for conda.

However, there are drawbacks, and these are summarized by this frequently heard statement from Zulip users:

I didn't like Zulip at first, but once I got used to it, I quite liked it.

Which says that Zulip is a good platform that is initially off-putting. And that's the challenge. We want a platform that encourages people to engage. Zulip does that, but only after a relatively steep learning (more of an unfamiliarity) curve. It's structured differently from platforms like Gitter and Slack, and that takes getting used to.

The driving question here is how many people would we lose before they learn the platform?

• Zulip Cloud Standard is free for open source projects.
• The public access option lets administrators configure selected streams to be web-public. Web-public streams can be viewed by anyone on the Internet without creating an account in your organization.
• Logged out visitors can browse all content in web-public streams, including using Zulip's built-in search to find conversations. Logged out visitors can only access the web-public streams in your organization, and the topics, messages, uploaded files, and emoji reactions in those streams.
• Web-public streams do not yet support search engine indexing. You can use zulip-archive to create an archive of a Zulip organization that can be indexed by search engines.
  • Well, that is kind of low budget web search support for chat/forum access.
  • For the forum it is essential that the posts get indexed, and that search results point into the chat/forum where people can start participating. Zulip does not give us this.

Discord

Discord is another widely used chat platform. It's also free. Downsides include that you have to join the channel to see content, and you need an invite to join.

• Discord an account to read content.
• It can't be indexed by search engines.
• Discord's internal search capability is not bad.
• Discord is available for free.
• Discord supports a large number of discussion rooms and private chats (as does Element, Gitter, and Slack)
• Has extensive moderation support.
• Many people will already be familiar with Discord.

Summary

We have several choices for a chat platform.

• Some of them are not readable unless you log in first.
• None of them are usefully indexed by Google (although some have more hope than others).
• Some have better search support than others.

Current Recommendation

Note: The "current recommendation" is whatever the current leaning is. Once discussion has settled, the current recommendation will be posted as the recommendation in the CEP PR. We will also not include the full discussion from this issue about different platforms that were not selected. The CEP will focus on the choice we make in this discussion.

First Choice: Element

Why?

• Slack's recent policy change makes it unaffordable to search chat history.
• Gitter is going away.
• Zulip has too steep of a learning curve.
• Discord requires invitation and login to see anything. It has no hope of being publicly searchable
• Element is free, has a future, is open source, does not require an invite or a login to reach content, and has a shallow learning curve.

Forum platform

Forum platforms complement chat platforms. Forums are particularly good at longer form questions, answering threads, and creating an online archive / knowledge base over time about the topic. All conversations are threaded in forums. However, there is more startup friction in forums than there is in chat. Typing a sentence or two in a chat platform feels less intimidating than creating a new post in a forum.

Discourse

Discourse is a widely used forum platform in the open source world. A hosted solution is available for free. Discourse is infinitely configurable and extensible and has good community management features. It will be familiar to many in our community, and will feel familiar to anyone who has used StackExchange.

• Is free for open source communities, with some limits:
  • 5GB total storage.
  • 50K / month page views, including web crawlers.
• Is indexed by search engines
• Has rewards based gamification (but not ranking gamification)
• Provides robust support for tagging and categorization of content.
• Many plugins are available (see Standard plugins)
  • This includes a plugin to post activity from Discourse into Slack, Discord, Zulip, Gitter, and Matrix.
• Familiar look and feel because of StackExchange and wide use of Discourse.

GitHub Discussions

GitHub Discussions is a way to have a discussion forum within the GitHub ecosystem.

• Free
• Indexed by search engines.
• Requires a GitHub login.
• Supports categorization of posts, including custom categories.
• Supports labels (tags)

See the conda Infrastructure discussion board for an example.

Zulip

Zulip, as mentioned above, is both a chat and a forum, or a forum that feels like a chat. See above for most pluses and minuses. One particular minus for Zulip as a forum is that posts are not searchable by search engines. This is a significant drawback for a forum because over time we want the forum to be a knowledge base. That doesn't work if Google can't find it.

Current Recommendation

First Choice: Discourse, followed by GitHub Discussions

Of these options, Discourse and GitHub Discussions ate the only choices because of Zulip's lack of search engine indexing. Discourse is a widely selected choice for communities like ours. GitHub Discussions may also be a viable option. Discussions is integrated with GitHub, but also does not appear to have the robust infrastructure and forum management options that Discourse has.

Email platform

What happens in chat and forums now, used to happen in mailing lists, and it was in almost all ways more painful in mailing lists. However, mailing lists still have a purpose as news distribution channels. Is there a new release of conda-lock or mamba? Is there an upcoming PackagingCon, or training about defining your tools in conda? Announcement lists are good for keeping the community up to date on that latest and greatest big news.

Announcement only lists have different requirements than discussion lists: There is no need to support threading and moderation for instance, as there is no discussion. Discussion happens in chat.

So, what are our options here?

Self Hosting

We could set up our own MailMan (MailMan2 please, not 3) or Sympa server. This gives us total control over the mailing list, and allows us to easily set up additional mailing lists when we need them.

However, there is currently nothing else in the plans that requires the conda Organization to host a live server that we are responsible for keeping up and running (and paid for). We are currently pretty sure that everything in conda.org can be run by GitHub. Mailing lists alone may not be enough justification to take on setting up and maintaining our own server.

Google Groups

Google Groups is another option. It has most of the advantages of having our own hosted mailing list server, but without all the drawbacks.

Google Groups is free to use, and we could probably even have the list name be announcements@conda.org.

There are limits, but those limits aren't published anywhere! Unless we want to pay for support, in which case they are published. It seems that the only way to understand the limits is to push them.

Other options for hosting?

Email Services

Services like MailChimp and Constant Contact offer very polished services for announcement only mailing lists. However, these services cost money, and polish is not really a priority for open source projects.

Options

There are a bunch of options out there for announce-only mailing lists. Here I present what the "best fit plan" looks like for a number of them. "Best fit plan" is defined as up to 2500 list members, and up to two emails per month (although I think that would be a heavy month).

It's also worth noting that it should be easy to migrate off of any of these services whenever we want to.

Provider	Best Fit Plan	Facts	Cost
Sender.net	Free Plan	- 2500 subscribers - 15k emails per month	Free
MailerLite	Growing Business	- 2500 subscribers	$15 / month
MailChimp	Free	- 2000 subscribers - 10K emails per month	Free
Constant Contact	Core	- 2500 subscribers	$35 / month
SendInBlue	Lite	- unlimited subscribers - 20K emails per month	$25 / month
Benchmark	Pro	- 2500 subscribers - unlimited emails	$30 / month
OmniSend	Standard	- 2500 subscribers - 30K emails / month	$35 / month

From these options, Sender.net appears to be the best. Its next step up is 5K subscribers for $18.25 / month, which is also better than the rest.

Current Recommendation

First Choice: Google Groups, followed by Sender.net if necessary.

Google Groups is widely used and can be configured to be announce-only.

Lean away from hosting a mail server ourselves. We can reconsider this if we ever need a server for another purpose.

What to do with existing channels?

All this work doesn't do any good if we aren't successful in moving people off of any of the channels we are trying to move people away from. How do we accomplish that?

Conda-Centric communications channels

Among the four conda-centric channels three can be combined into a chat and a forum channel. GitHub issues is the one channel that should stay in use as is.

Conda Gitter and Conda Slack should be combined. If we stay with either Gitter or Slack, then it's a matter of shutting down the other one and encouraging users to migrate. If we favor of a new platform, then we would shut down both and encourage migration.

The conda Google Group would be shut down and we would encourage members to use the new chat and forum platforms, and to subscribe to the announcements mailing list.

Nearby communication channels

For the nearby channels we don't need to do anything. But we should do some things like

• reach out to channel owners to let them know what we are doing, and seek their advice.
• let them know that they are welcome to join the conda channels now, or later, or not at all.
• Announce when new conda channels are ready. Stress that they are not replacing the nearby channel, but rather are for conda-centric discussion.

Across Town communication channels

The across town channels exist for reasons that are completely independent of conda. We can't make changes to these platforms, even if we wanted to. But, we can change how we recommend the community responds to items posted to these channels.

Depending on the channel (some examples):

• Twitter - usually point people to official channels, sometimes pointing them to where this question has been asked before. Twitter is not a great discussion channel
• Reddit, StackExchange, similar - Sometimes answer in place. Always include pointers to official channels

Resolution

Summarize our decisions here. Which we will do, once we create the CEP.

[jakirkham]

Thanks for the detailed write-up Dave! 🙏

Maybe another one to add to the list would be GitHub Discussions. This is also being used for conda/infra.

Something that gives me pause with any platform is whether I need a new login (or whether say GitHub can be used). Not sure if the same is true for others, but it does seem like a potential point of friction (should add this seems to keep growing over time with 2FA, recover keys, mobiles and other info these accounts need for maintenance). Can see value in having a separate login, but an easy on-ramp can be quite helpful here.

With the mailing list, if our only goal is announcements, we could consider using an RSS or Atom feed. Or we could fold this into the discussion forum using a designated topic thread.

The last thing I would add is I don't think we need to centralize all communication (nearby, across town, or wherever else people like to talk to each other). Though we could benefit a lot from having a clear page with a list of which platform to use and for what purpose. That's missing today.

[jaimergp]

This is such a thorough compilation, thanks Dave!

First, I'd like to emphasise one point. Whatever we choose (more on this later) will have to be featured visibly on conda.org. It deserves its own first-level header ("Community" or something like that), where the discussion places are enumerated, described and linked to.

Now, some notes about my prior research on this:

Forums

• One of the things I find missing in our community is the "forum" type of discussion platform. As in "structured threads of information that can be indexed by search engines". In this regard, I think we all agree Discourse is the way to go, with no close contenders.
• Once the "forum" is covered and visibly announced, I hope the chat platform is not used as much for long support discussions or decision making (where most of the information value is deposited). This would make the search requirements for chats less strict.

All of this to say I really really want a Discourse instance for the conda community, hopefully joined eventually by the satellite communities like conda-forge or bioconda.

Chats

• Discord does not always "require" an invite, in the sense that the invite link can be 24/7 available on a website (it can be something memorable like discord.gg/conda too). That said, it's not searchable. But maybe chats do not need to be searchable from Google, just from the platform itself. Discord search capabilities are not bad. I love Discord for its UX and modding tools, but it's privately owned so... who knows what will happen to our data
• Slack is a close second UX-wise, also privately controlled and with severe limitations.
• Gitter is ok. Too simple for my taste (where are my emoji reacts!), and almost no modding tools. I guess it gets the job done.
• I am assuming Element is like Gitter but a bit better? I see emoji reacts on the screenshots. I tried using Element to access conda-forge's rooms and it was not obvious and required a login to even read the contents. Maybe this can be configured, I don't know.

I guess I'd be ok with Element as long as getting access to it is super simple and obvious: you click the link, you see the room with no login required, whether you have the app or not.

[tnabtaf]

@jakirkham and @jaimergp Thanks for the input. Some comments on some of the specific input.

From @jakirkham

Maybe another one to add to the list would be GitHub Discussions. This is also being used for conda/infra.

Something that gives me pause with any platform is whether I need a new login (or whether say GitHub can be used)...

John, I have added GitHub Discussions to the forum option list. It's currently listed as the second preferred option, but if people like it more than Discourse, then we will bump it up.

The "new login" statement made me question my assumptions. In my previous community, the fact that a GitHub or Twitter login was required to use Gitter was irritating: Much of that community would not have either. However, for conda, the opposite may be true. Most of our community probably already has a GitHub ID.

Discourse does support authentication through Google, Facebook, and Twitter by defaults:

If you want to use Google, Facebook or Twitter, those are included out of the box

It's not clear to me if GitHub authentication in Discourse is a paid or just optional product.

From @jakirkham

With the mailing list, if our only goal is announcements, we could consider using an RSS or Atom feed. Or we could fold this into the discussion forum using a designated topic thread.

I still argue for an announcements mailing list. Feeds can also be supported (via conda.org) but I think the easiest way to get announcements to our broader community is a mailing list.

From @jakirkham

The last thing I would add is I don't think we need to centralize all communication (nearby, across town, or wherever else people like to talk to each other). Though we could benefit a lot from having a clear page with a list of which platform to use and for what purpose. That's missing today.

From @jaimergp

First, I'd like to emphasise one point. Whatever we choose (more on this later) will have to be featured visibly on conda.org. It deserves its own first-level header ("Community" or something like that), where the discussion places are enumerated, described and linked to.

@jakirkham I think we agree (mostly) here. I don't think it's even possible to centralize "across town" discussions.

And +1 for clearly documenting channels on conda.org.

From @jaimergp

Forums

Yes. :-)

From @jaimergp

Chats

I view Element as noticeably better than Gitter, but not as good as slack.

And I updated the description of Discord to include your points.

[tnabtaf]

Hi All, it seems that Discourse for the Forum is the least controversial part of this proposal. Unless I hear objections in the next two days, I will start investigating that option with Discourse. Note that this does not mean that we are committed to Discourse for the forum. That commitment won't come until the resulting CEP is proposed and voted on.

[tnabtaf]

An update:

• I have submitted a request for a conda.discourse.org Discourse forum.
• I have explored Element further. So far I have found only one drawback that compares unfavorably with Gitter (but favorably compared to Zoom). I will keep exploring next week.

[tnabtaf]

An update:

• Discourse approved our request. :-)
  • Working on configuration now
  • Would love to hear any thoughts people have on what our categorization scheme should be.
• Have spent even more time with Element. I think we should pursue that for chat.

[tnabtaf]

Hi All, I have created a skeleton for a conda Discourse forum. I am hoping we can launch this by the end of this month. But before that I am seeking your help!

• How does the categorization / hierarchy / structure look?
  • How are the categories? How are the subcategories?
  • Should some of the subcategories (conda, mamba, etc) be dropped and we just use tags?
  • Would you like to have subcategories for other components (especially want to hear from component developers of things like conda-lock and grayskull)
  • Package Building
  • Not at all sure about this one. Are most package building questions specific to the channel they are being built for? If everything is channel specific, then the proposed structure may not work.
  • I am also reaching out to conda-forge and Bioconda for input.
• Each category/subcategory we decide to keep would benefit from having a pinned "Resources" post that lists useful resources for the topic.
  • please let me know if you are interested in writing some of those and I will send you an invite. These could also be written by committee and then posted.
  • eventually, the bulk of these resource pages could be moved to conda.org.
• We need more site admins and moderators. Right now it is just me. Let me know if you are interested.
• What posts should we pre-populate the site with before we fully launch it?
  • This makes the site more inviting in the early days.
  • I added the conda 4.14.0 and conda 3.22.0 release announcement. I suspect we should also add some CEP announcements. Let me know if you want to post something, and I will invite you.
• What else?

[tnabtaf]

Planning to announce the conda Community Forum after today's community call. I will also open up the site to new user registrations.

[jakirkham]

Just a note, the existing Gitter channels are already mirrored to Matrix (so available via Element). So there shouldn't be anything we need to do there per se (except maybe updating links).

[BastianZim]

Hi, apologies for posting this here but I'm not sure where else to? :)

Currently, the discourse uses @tnabtaf personal login when authenticating with GitHub. I know that you're managing it so that's fine but it might put off new users? Not sure how others manage this but it might be good to move this to an official and verified account.

[jezdez]

@BastianZim Good call-out and we can fix that, @tnabtaf let me know how I can help with this. We've used the dedicated @conda-bot account for generating OAuth/personal access tokens before as a way to centralize these things.

[tnabtaf]

@BastianZim Thanks for pointing this out. I had no idea I had inserted my account into the authentication process. I will investigate today.

[BastianZim]

No problem! If I remember correctly, there is even a way to get a checkmark verifying the account/organisation but I might be misremembering this as I couldn't find anything googling.

[tnabtaf]

I take it back! I did the right thing for Google authentication, but this was a temporary workaround for GitHub that I promptly forgot about. :-( Thanks for the reminder. I will work with @jezdez (who has access to GitHub magic) today to fix this.

Thanks for your patience.

[tnabtaf]

@jezdez has created this issue to track this.

[tnabtaf]

Hi All,

I am going to declare victory on this issue.

• We have started using Sympa for mailing lists
• Slack has been closed and topics migrated to Matrix.
• Gitter is now fully integrated into Matrix.
• All known conda-ecosystem related Matrix rooms are now available from the conda matrix space
• The conda Discourse site is up and running for Q&A

Thanks everyone for helping make this happen.