Begin version 0.6: more support for project authors to say things about packaging

[ceball]

Currently, pytdev expects python projects to use python setuptools config to declare as much common packaging info as they can (e.g. description, dependencies, etc). (Note: Although it's setuptools setup.cfg right now because support for that's all built into python and it's used anyway by conda build, it could instead be any other format, e.g. conda build meta.yaml.)

Whatever's used, though, it's probably not possible for project authors to express in only one packaging tool's existing config everything they know to help packagers. This PR allows projects to add some custom things to setup.cfg, so that project authors can give more hints to packagers. pyctdev will use those hints when packaging (where possible, depending on what kind of packages it's making). And manual/semi-manual packagers (e.g. conda-forge package maintainers) could read those hints.

New features

• Optional dependency version pinning. E.g. to optionally generate pinned packages and/or environments containing versions of dependencies that are known to be mutually compatible. (Conda only right now; should be added to pip ecosystem too.)

• "Namespaces". E.g. because pypi name doesn't match conda main ("defaults") name which doesn't match conda-forge name. https://github.com/pyviz/pyct/issues/42.

• Declare non-python (non-pypi) dependencies. E.g. a dependency not on pypi yet, or a lower level ("system") dependency that is unlikely ever to be on pypi. Python project authors may still know about constraints to be applied where possible e.g. with conda, apt, etc. (Currently only conda; could support other package managers in future.)

• Support for multiple packages where the packaging system isn't like pip/pypi. E.g. conda default package might include everything, while pip/pypi default is minimal dependencies. Project can declare mapping of packages to python extras.

• Can generate environment files from info in setup.cfg (not just actual environments). Can pin and filter dependencies (from info in setup.cfg). Note: currently only conda ecosystem; should be added for python world too.

[jsignell]

@ceball I'd like to merge this since I think it does some nice cleanups. Could you check off which of the goals listed above this PR accomplishes?

[jsignell]

I was wondering if we should just vendor in doit as well as tox so that we can just have one dependency and overwrite CmdAction and PythonAction with more logging.

[jbednar]

Fine by me.

[ceball]

I was wondering if we should just vendor in doit as well as tox so that we can just have one dependency

Maybe re-write pyctdev using something other than python and distribute as a static binary? ;)

In case it helps to know the history, I vendored tox (and one of its required dependencies, virtualenv) to allow part of tox to be used by conda users. The readme in the _vendor dir says:

The zip files here allow tox to be used from the conda ecosystem for reading tox.ini; 
see imports in pyctdev/util.py for explanation.

pyctdev/util.py says:

# Fallbacks for conda, which can't install tox from defaults (at
# least, not as of June 2018). Allows to parse tox.ini (i.e. to use
# tox.ini as single place where all test cmds and envs are stored)

I don't know how much of that is still true.

I remember simplifying the vendoring a bit in 0.7, but I don't remember the details.

According to https://www.wheelodex.org/projects/doit/, doit has these requirements:

Requires-Python: | >=3.4
Requires-Dist: | cloudpickle
Requires-Dist: | macfsevents; sys_platform == "darwin"
Requires-Dist: | pyinotify; sys_platform == "linux"

Not sure how feasible to vendor those things/their dependencies, though the parts of doit used by pyctdev (at least currently) presumably don't need inotify/similar.

[ceball]

I was replying to your comment as it just showed up in my email. But I see now it's from 12 Feb! Sorry - just ignore me. Evidently I don't understand github emails any more :)

[jsignell]

Oh it probably showed up because I pinned pip on this branch :)

[ceball]

I'm planning to merge this PR, but then need to check that metadata in setup.py is still supported.