Lack of or Rude Communication

[dstufft]

A user has expressed concern that the folks working are either not communicating well (Rude) or are just not communicating at all (lack of responses to threads, issues, etc). How can we foster a welcoming space and ensure nobody feels ignored?

[qwcode]

Part of the problem, is that when you get down to it, there's really only about 5 people doing steady work (PEPS and/or coding) in packaging right now (in the "mainstream" projects that is). 5 people can't respond to all the issues that come up.

[ncoghlan]

The other part is the generally poor documentation, and the fact that the current state of a lot of things isn't defensible in an absolute sense. The history of "Hey, here's how we're going to fix it!" -> "Oops, that didn't work :(" in the packaging space also makes us wary of overpromising and underdelivering (again).

[Ivoz]

I think a major problem is that there has never been a clear, central, canonical open line of communication for packaging issues to the wider python community.

So after people learn how to deal with x broken issue or z feature in their daily lives (and assume that will work forever more), and a year down the track we start trying to fix it, or break something in the process of trying to fix things, or improve security such that y no longer works, the first experience they have with us is that we've broken something of theirs - they never get a chance to start off with a positive experience of being informed about what's happening, or even submit input about anything they don't agree with or want to ask questions about.

A smoking gun ATM is setuptools briefly dropping Features to many people's surprise and chagrin.

TL;DR a lot of people start off communication with us from a negative experience, so it's already hard to turn that around. This would be a lot smaller problem if we could look for ways to change that defacto situation.

e.g, this is the right idea, we simply need far more of that that can reach more people.

[daenney]

As someone who's only recently hit the Python packaging space:

• The docs. I think we've been over this plenty of times.
• There is no clear entry-point for communication. I got pointed to the Python SIG mailing lists when I submitted an issue on Github that was a question because I couldn't find another way in. Most docs wrt setuptools, distribute or pip-installer don't mention this SIG either. Also, not everyone knows what SIG means, I actually had to look it up, only then did it make sense to me.
• IRC is a nice, low bar entry into a discussion. It's also nice to be able to lurk in a channel and just see what's going on. For quite a few of us 'youngsters' mailing lists are a fairly archaic way of communication, a more real-time approach is often preferred.
• Python.org > Community > IRC was the only place I was able to find a reference to #distutils
• Mailing list archives are fairly terrible at being searched and browsed. It would be nice if some of the comment threads that keep coming up are extracted into some kind of site/FAQ thing.
• Python.org's SIG page is pretty empty: http://www.python.org/community/sigs/, are there no active SIGs anymore and if not, why isn't there one for packaging?
• Apparently there's a distutils SIG but the only place I could find out it existed was by searching https://mail.python.org/mailman/listinfo.
• The name distutils is being used everywhere but just about no-one knows that's what it's called. When I search for information about things like this I'll likely try the terms python packaging, setuptools and/or distribute, not distutils.

[ncoghlan]

packaging.python.org is designed to be the new hub for information. The Python 3.4 docs will redirect users to that in preference to reading the low level technical details in the stdlib docs (pending some work to actually make those docs changes).

We're aware of the mailing list discoverability issue, and I expect python.org will eventually be upgraded to Mailman3 and the HyperKitty web gateway.

[dstufft]

I tried to argue that we should close down distutils-sig and rename it packaging but people yelled at me for it :(

I think packaging.python.org is good, but I don't think it makes much sense as a changelog or announcement service.

[qwcode]

"packaging-release" list? for release related emails for any of the pypa projects?

[ncoghlan]

I expect we can get access to python-announce for packaging related announcements, especially now Python 3.4 bundles pip.

[dstufft]

That could probably be reasonable. I'm thinking in terms of PyPI too, there's not really a great way for us to say "Hey in X time/releases Y is going to break/change" and ge that message out to the wider population.

[daenney]

We're aware of the mailing list discoverability issue, and I expect python.org will eventually be upgraded to Mailman3 and the HyperKitty web gateway.

What about in the mean, now, time?

[merwok]

I suspect the configuration set up to serve the devguide could be reused to have docs.python.org/packaging.

[ncoghlan]

On 8 Mar 2014 03:58, "Daniele Sluijters" notifications@github.com wrote:

We're aware of the mailing list discoverability issue, and I expect python.org will eventually be upgraded to Mailman3 and the HyperKitty web gateway.

What about in the mean, now, time?

It's not currently a priority for the infrastructure team, as Mailman3 is not yet released.

Reply to this email directly or view it on GitHub.

[daenney]

Oh, another one, there's @pypa on Github and on BitBucket, both seem to be in use and have a different set of repositories.

[dstufft]

yes those are both us owned by PyPA, some projects are using git and some are using hg,

[hickford]

My experience: GitHub (and Bitbucket) are great. Search engines find relevant discussions, or you can browse the repo to see everything. Everyone's comments are on one page, and it's obvious how to add your own (there's a giant green 'comment' button). It's dead easy to make an account--most community people already have one--but if not, it's a familiar process. Thus, GitHub is welcoming, and levelling too.

Mailing lists, on the other hand, are dire, and I believe excluding to almost all of the community. From a search engine, you'll be taken to a single post from the middle of the discussion. You might see excerpts from earlier posts in reverse chronological order. It's disorientating. To read all the comments, you have to click around a tree structure (shown on a different page). That is ludicrously unfriendly. Most people surely give up. Perhaps the experience is better if you are a member of the mailing list, and receive the messages in your inbox. Understand though, that new people will always read mailing list posts on the web.

Worse, there are no instructions on the page how to comment. For example https://mail.python.org/pipermail/distutils-sig/2013-August/022529.html , there's no big green comment button. If you click around, you can sign up to receive future posts in your inbox (not sure I want that) but that still doesn't explain how to reply to the post you're reading. I suspect older community are oblivious to this problem, because they grew up on mailing lists. Understand, most young people don't know how to use a mailing list--they've never seen one. By modern standards, the process to do so is laughably slow. (I'm reading a post on the web. I should be able to write a comment from the same page). It's prohibitive for many.

PEPs are thus problematic. For example http://legacy.python.org/dev/peps/pep-0453/ . There are no community comments on the page itself. There are links to five different mailing list pages where you can read what other people have said in the past, but there are no instructions how to have your own say. That's a shame. (Compare with Ruby feature requests which are discussed on an ordinary comment thread on the same page https://bugs.ruby-lang.org/issues/8992 )

bugs.python.org is also unwelcoming, in my experience. I reported a bug once, it was closed immediately, with a reply

You can't comment here, you're in the wrong place. This subject been discussed on the mailing list several times before [no link given]. If you want to comment, you should join the mailing list, go back in time, wait for the right post, and reply there.

The message was polite, but completely unhelpful. I felt unwelcome. I gave up trying to contribute to (what I assumed was) the community and started writing rants on Stack Overflow instead. They proved extremely popular, which encouraged me to try participating again. I'm happy I found your GitHub group--you have interesting things to say, and you've accepted my contributions. Thanks. I've even submitted pull requests--I love how levelling GitHub is.

To clarify, I don't believe the poster on bugs.python.org was trying to exclude me--they probably thought they were being helpful. But the perceived "we do things our way" attitude made me feel ignored and unwelcome. I'm sure other potential contributors have turned back at the same or earlier hurdles. http://bugs.python.org/issue16675

[hickford]

It's not just me who loathes mailing list. See http://programmers.stackexchange.com/questions/71148/why-do-programmers-still-use-mailing-lists . These are technical, competent, experienced programmers speaking. Woe for young programmers about to make their first contribution to open source.

• Being a new graduate, I find it much easier to communicate using forums rather than mailing lists. I suppose the older crowd is more comfortable using what they are used to?
• Mailing lists is one of a few things which I was never able to understand. It is simply unusable in my eyes.
• Here, here. I HATE mailing lists, unfortunately the nature of my work requires me to use them occasionally
• This question is the best question i've seen since the creation of this domain. I think people should really start living in the 21 century... instead of the 17 century

It's a real issue, but you won't read about it on a mailing lists.

The answers to the OP's questions are hopelessly unhelpful. "I find mailing lists easy to use, so you must be an idiot." and "Trees are trivially the natural representation of human discourse. (My mail client flattens them to help me read them)".

Jeff Atwood is right http://blog.codinghorror.com/web-discussions-flat-by-design/

threaded discussion is ultimately too complex to survive on the public Internet.

[ncoghlan]

The mailing lists aren't going anywhere - web forums are appallingly bad for high volume communication, and the preferences of existing contributors take precedence over newcomers on that one.

However, expect Mailman3 to be deployed at some point in the next several months, which is aimed directly at addressing the relatively high barrier to entry that mailing lists can pose for web focused users.

[dstufft]

I don't know much about Mailman3, but there's relatively little reason why you can't present a forum interface to a mailing list.

[ncoghlan]

That's exactly what Mailman3 (technically, the HyperKitty archiver) does. The other way of viewing it is as a web forum with email integration that is actually usable.

[Ivoz]

In my own experience I find forums far easier for high volume than mailing lists. For one, they often have many subcategories to allow me to focus on only the content I'm interested in; otherwise they have a single stream of easily scrollable flat messages per thread that is far easier to digest. Either that or Thunderbird or the way I have it configured is just terrible for reading through them. That and the endless, varied, non-back-linkable quoting.

[daenney]

I honestly don't believe mailinglists are a high enough barrier to prevent people from contributing. I've been put back by them a few times too but that's never stopped me.

Reading your comments I don't believe you have an issue with mailinglists per-se but more a UX issue. E-mail is actually a very good format to receive and store these kinds of messages in but most of the UI's around it suck. Google's proven that with Groups, which are just mailinglists, but with a flat hierarchy and an inline editor. It has become an extremely popular way to host mailinglists and make them easily accessible and searchable.

Another advantage that the mailinglists have over most of the comment-based systems is that they are immutable. Both on Stack, Github etc. people can come back later and change their respones essentially derailing half a conversation pretending the other part didn't happen. That's extremely unhelpful down the road and bad for archival purposes.

Mailing lists also have the advantage that due to the slighly highe barrier of entry you don't get flooded with ridiculous amounts of inline gif links or stupid comments whenever an issue makes it to 4chan or HN. They're a bit of a double-edged sword.

There's an enormous amount of projects around, including incredibly succesful ones like Python and Debian, that rely on mailing lists for most communication, discussion and contribution and that's been working out really well for them for more than a decade. Perhaps newer generations prefer a click-and-rant interface but there's nothing stopping us from providing both.

@ncoghlan mentioned HyperKitty a few times and I really do suggest you have a look at the demo server: https://lists.stg.fedoraproject.org/archives/. It's already far easier to work with than current systems (albeit a bit slow). The UI needs some work but it's a good step forward. I believe to only thing it's missing is an ElasticSearch backend to provide easy full-text search but I'm sure that'll get added to the mix in due time.

[hickford]

You're right, with a better web interface, mailing lists needn't be intimating. Google Groups is excellent because it enfranchises web users. It shows the conversation on on a single page, and adds a big red 'reply' button. People participate who were previously second class read-only citizens. If Mailman were to add this functionality, and it were adopted, that would be fantastic.

Yes of course successful projects can be run on mailing lists. The value of making them more accessible would be to widen the diversity of contributors. I believe this is a worthy goal of itself, though I like to think it might also result in better quality software.

The success of GitHub shows that web user are capable. For example, Pip (developed on GitHub) has received 2800 commits from 140 contributors. https://github.com/pypa/pip . Some of these people will have never posted to a mailing list, or submitted a patch in the traditional manner.

(It would be interesting to compare the commit to contributor ratio for projects developed in different spaces).

[gnat]

Forums are passive and do not clog my inbox: Forums enable me to browse and choose when to pull data without committing my mailbox to it. Mailing lists are inferior because: I have no choice: everything pushed into my mailbox if I subscribe.

Google Groups is a barely acceptable "middle of the road" solution, but as an open source contributor myself, I always think twice before subscribing to yet another mailing list and generally opt not to unless the contribution requires me to do so. I usually unsubscribe shortly after.

[rsyring]

See also: https://github.com/python/psf-infra-meta/issues/1