astropy-helpers

[stscicrawford]

There is some debate on astropy-helpers and whether or not to base our packages on it -- see the discussion here: https://github.com/spacetelescope/stsci-package-template/pull/16 . As that is an intermediate issue, opening this ticket here for further discussion about it.

[eteq]

Good timing: I was just about to post in spacetelescope/stsci-package-template#16 ... so see there for the context of this message (cc @sosey @arfon)

An important bit of context regarding the helpers: part of the motivation was to lower the mental burden on scientists who aren't going to take the time to become deep experts in packaging. i.e., they can learn how to write docstrings that get rendered in sphinx without knowing much about sphinx, or unit tests using remote data without really understanding pytest or the details of the remote-data plugin. By creating shared infrastructure for that it's easier for the community to share support for such things, so people who are experts in the shared infrastructure can support a larger group of the community.

But all that goes out the window if the audience are all people who already experts in such things. In my experience that's the crowd that is more likely to be 😡 - if they already know how to do this stuff so they see the helpers as getting in the way. So that leads to the second motivation of the helpers: to provide shared infrastructure for these things so that at least we share the maintenance burden instead of everyone rolling their own. I would argue that this is still a worthwhile goal, but if it really is too much of a burden that's an argument to simplify the helpers machinery, but not to throw it away completely in favor of something else.

So I guess I don't agree with the last part @jhunkeler says: we can't say "use a different template if it's not astro-related" - it's all astro-related because we are producing software for the astronomy community. We end up paying for it in the end anyway if we can't lower the maintenance burden on the folks who are 😡 about the template.

So that implies we need to fix whatever needs fixing about the template. There's already been some work making it "less astropy-specific", but would be great to know exactly what needs changing for it to meet the needs here.

[mperrin]

I fully agree with the goal of "lower the mental burden on scientists who aren't going to take the time to become deep experts in packaging".

I'd love for this to become a practical way to "OK, so you're starting some new package? Well, voila, now you immediately have a framework with all this useful infrastructure like Sphinx and Travis and code coverage tests". As you say, experts in the shared infrastructure should be able to support a larger community.

For the experts, somewhere there needs to be a succinct summary/description of what functionality the template provides, what behavior it changes, and why those choices were made. Minimizing surprises should be one of the goals. My complaints over the years about the package template have largely been from times when it led to surprising behavior or changes that I was not expecting. I also think there could be some cases where the helpers machinery could be simplified. (Just to take one example, the astropy-helpers .travis.yml file sets things up for c instead of Python, and has a bunch of stuff about cython and c compilers; this is all totally irrelevant extra fuss for the common case of a pure Python package. Yes that's necessary for building astropy, no it's not for the case of a general-purpose STScI package template, since our hypothetical non-expert-developer is probably unlikely to be wrapping C code)

[mperrin]

Thinking about this more, perhaps the package template could come in two flavors?

• A streamlined version set up for pure Python packages, which fills in reasonable defaults to as many as possible of the cookie cutter questions (helping to address @sosey's point that many of the cookie cutter questions will be inscrutable to non-experts).
• And a more fully-featured one that opens up the options for the extra complexities, for all those who are expert enough to know they want or need more functionality than in the streamlined template.

I don't know enough about cookiecutter to know whether something like this could be done with two variants of an install script pointing at the same template, perhaps.

[pllim]

APE 17 is accepted, so astropy-helpers is no more and this is now a non-issue.