Update standards based on Nov 2019 PyHC meeting discussion

[namurphy]

This pull request is to include changes to the community standards based on the discussion at the PyHC meeting from November 2019. We probably still need to update the author list to reflect the people who were in attendance in Boulder. We'll need to review this during a PyHC telecon before we accept this, which will probably be in January 2020.

Here are the notes from the meeting back in November:

• Change: All functions, classes, and modules in the public-facing API...
• Question: what do people think of Black?
  • It’s opinionated and consistent with the standards
• Add actual links for “System[link]” and “acceptance[link]”
• Python 2.7 has about a thousand hours left
• Change: “clearly and constructively explain when a contribution is not accepted”
• Change in #15: “Projects should have a reporting procedure for violations of the code of conduct (see [link to NumFOCUS’s for ).”
• Add a bullet point
  • Be explicit on what versions of dependencies a package supports.
  • Strong recommendation that we should support NumPy Enhancement Proposal 29 which provides recommendations on which prior versions of Python and NumPy to support.
  • “Should” is fine
• Do we want to figure out a way to “grade” packages on the PyHC website?
  • Making an assessment can be really useful to packages
  • Probably need someone(s) to go through packages
  • Goes back to governance
  • Self-assessments are a first step; and are important
    • Maybe have a template for that particular type of PR
  • Maybe if someone wants to add their project to PyHC’s website, then they should provide the results of their self-assessment
  • Astropy's affiliated packages list has a “traffic light system” (red/orange/green)
  • Would be wonderful to have community reviews; really helpful for packages
    • JOSS is a good model for this
  • Should current packages plan on going through this procedure, doing a self-assessment?
  • We’ll need to first figure out what the items to be evaluated on: what the metrics are
  • Does anyone want to volunteer?
    • Eerie silence
  • Bring up at next telecon
  • By the next meeting in the spring, aim to have all packages on the website reviewed

[namurphy]

If you approve these changes and would like to be listed as a co-author, please let me know! You can add a comment, suggest a commit, or approve this pull request. It'd be helpful to know the name/affiliation you would like used if they're different from the original document, as well as your ORCID if you have one.

Many thanks to @ehsteve and everyone else who shepherded the process the first time around!

[namurphy]

We also have been talking about who is willing to volunteer for going through PyHC packages and evaluating the extent to which each package meets the standards. We could potentially do this during the hackathon following our spring meeting in Cambridge. The main meeting will be from April 27-29, with the hackathon beginning on the afternoon of the 29th and lasting ~1-2 days.

[wtbarnes]

My affiliation should be updated to NRL and my ORCID is 0000-0001-9642-6089.

[wtbarnes]

We also have been talking about who is willing to volunteer for going through PyHC packages and evaluating the extent to which each package meets the standards. We could potentially do this during the hackathon following our spring meeting in Cambridge.

I wonder if this could be a responsibility of the proposed PyHC developer position? At least the first pass and then the initial review could be further evaluated by the community.

[namurphy]

I think it would always be better to have discussions of PRs on GitHub so that they are "in the record" and everyone is able to participate.

Good point! I had thought I had emailed the list after I originally made the PR, but I forgot to... I put my notes from today's discussion in the commit log, though I should have put it as a comment here. For discussions at meetings/telecons, we should make sure that notes are posted in the corresponding PR.

One topic of discussion during this telecon was on adoption of NEP 29 (which created a community-wide standard for how long to support older versions of Python and SciPy) by PyHC packages. Stuart Mumford suggested that it be okay for some packages to have stricter standards, and that packages can support older versions of NumPy and Python if they so choose. Michael Hirsch brought up that usage/download statistics from a package that supports Python 3.5+ showed that very few people are using Python 3.5 at the moment (and also that Python 3.5 gives the most trouble for the testing matrix). I brought up that core packages like SunPy should follow the guidelines of NEP 29 most closely, while packages that are currently under development and have few users would be okay in supporting Python 3.7+ or even 3.8+ if the development phase is going to last ~1-2 years. We decided that using "should" leaves open enough leeway for us to accept packages that have different requirements. I forgot to mention that I think that we should follow the same guidelines for Astropy (and ideally packages within PyHC) as we do for NumPy.

(These notes are a convolution of what people actually said, and my ability to remember 42 minutes into the past.)

[scivision]

minimum Python version observations

I have required Python 3.6 for over a year on almost 100% of my packages (aggregate millions of users across geoscience, life sciences, aerospace, etc) with only a handful of complaints, including from incorporation into Linux distributions, offline/proprietary/OEM users and others not captured by PyPi.

• PyMap3D (230K total PyPi downloads) has < 1% Python 3.5 PyPi downloads. PyMap3D still supports Python 3.5 to support OEM with long dev cycle on embedded systems.
• Numpy is down to about 15% Python 3.5 PyPi downloads, and that may start dropping soon as Python 3.5 won't be supported in the next Numpy release.

Although there are limitations on relevance of PyPi download stats (does not account for conda and apt install etc), this script allows easy plotting of last 6 months of PyPi download data over time: https://github.com/scivision/pypistats-plots

As Jeff also noted, there are increasing penalties to supporting Python 3.5. If one looks at and assumes sufficient representation of PyPi stats, Python 3.5 may be a small audience for a growing number of PyHC packages. In my opinion the main penalties of Python 3.5 include:

• declining CI support
• rapidly declining package support
• lack of newer subprocess features beneficial to managing tricky external processes
• lack of complete type annotation -- which helps catch bugs new and old
• Python 3.6 f-strings help avoid print() mistakes

Python 3.7 brings dataclasses and better UTF8 locale support among other benefits

[scivision]

Re: Black, I think it may be too much to have in PyHC standards in general, I have run into its corner cases that make code less readable. Also the whole defaulting to double quotes thing is problematic to me, although black -S is a workaround to retain single quotes.

I would rather have a statement encouraging setup.cfg over setup.py if we're going to get technical :-)

[scivision]

re: Docs in my opinion this should include as OK even those that use non-readthedocs sites like GitHub Pages. I don't recall the source, but I read several months ago that packages of mine that do use numpydoc extensively with GitHub Pages were marked as not having docs because I don't use readthedocs.

[blalterman]

Please add me to the list. Orcid is 0000-0001-6673-3432

[Cadair]

On line wrapping, I generally find the best thing is one line per sentence. It (ab)uses the git diff algorithm to highlight changes on the sentence by sentence basis.

[blalterman]

This is what I do and it even works with diffing LaTeX documents.

On Fri, Feb 7, 2020, 11:00 Stuart Mumford notifications@github.com wrote:

On line wrapping, I generally find the best thing is one line per sentence. It (ab)uses the git diff algorithm to highlight changes on the sentence by sentence basis.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/heliophysicsPy/standards/pull/16?email_source=notifications&email_token=ADB5MVM76IGJIIWJTBJN5HDRBWASJA5CNFSM4J4QTEJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELDQUAI#issuecomment-583469569, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADB5MVMOW4QKTEFFOOBOFN3RBWASJANCNFSM4J4QTEJA .

[asreimer]

The HDEE Step-2 are due 1 July. Is it expected that the PyHC standards will be updated before the due date?