Plan to support pep517/518

[ionelmc]

So a new plan to support them, based on discussion from https://github.com/tox-dev/tox/issues/1131. The intention of this plugin is to support platform specific building. So in the idea that this one day might find its way into tox's core please give criticism if you have any @gaborbernat @obestwalter @asottile @nicoddemus

So first, to set a record straight: I'm an ardent proponent of using concrete dependencies (aka version pins). Some more thoughts on that in https://github.com/tox-dev/tox/issues/1131#issuecomment-452735602 or https://caremad.io/posts/2013/07/setup-vs-requirement/ so probably not lucrative to argue with me on that 😁

I would like to support these sort of configurations (with the intention of keeping implementation simple and orthogonal, with a seamless transition from/to legacy non-pep51x wheel building):

A. Simple usage

[tox]
envlist ={py27,py34,py35,py36,py37,pypy,pypy3},
wheel = true
wheel_clean_build = true

B. Separated build venv (universal)

[tox]
envlist ={py27,py34,py35,py36,py37,pypy,pypy3},
wheel = true
wheel_clean_build = true
[testenv]
wheel_build_env = build
[testenv:build]
deps = setuptools_scm==123

C. Separated build venv (platform specific)

[tox]
envlist = {py27,py34,py35,py36,py37,pypy,pypy3}-{test,build}
wheel = true
wheel_clean_build = true

[testenv]
deps =
  build: setuptools_scm==123
  build: cython==123

[testenv:py27-test]
wheel_build_env = py27-build
[testenv:py34-test]
wheel_build_env = py34-build
[testenv:py35-test]
wheel_build_env = py35-build
[testenv:py36-test]
wheel_build_env = py36-build
[testenv:py37-test]
wheel_build_env = py37-build
[testenv:pypy-test]
wheel_build_env = pypy-build
[testenv:pypy3-test]
wheel_build_env = pypy3-build

Of course that is quite some boilerplate so with little effort I think substitutions could be supported for that setting:

[tox]
envlist = {py27,py34,py35,py36,py37,pypy,pypy3}{,build}
wheel = true
wheel_clean_build = true

[testenv]
deps =
  build: setuptools_scm==123
  build: cython==123
wheel_build_env = {envname}build

Here I'm not sure if I'm breaking the existing conventions about substitution - are they lazy or not normally?

Some details on how I would implement dependency handling will follow but lets discuss the configuration first.

[gaborbernat]

I think we should always have separate build. In general case the build envs should be just a suffix of the test env it builds for (e.g. py27 -> .py27-build). I also don't think we need build dependencies, all those should be read and parsed from pyproject.toml. Once we have the general case we can try to add some heuristic optimization for universal wheels. 👍 (aka single build)

[ionelmc]

I would allow mixing test and build into the same venv for 2 reasons:

• I already do it, and haven't shoot my foot yet (tho I don't use tricky build things like setuptools_scm)
• its faster as you don't have to build 2 envs

I think it's an acceptable compromise, or it should at least be allowable if not the default (eg: make wheel_build_env mandatory, but allow wheel_build_env = {envname} to emulate A)

For pyproject.toml yes, deps won't be mandatory in tox.ini. The proposed configurations are supposed to support both legacy and pep51x (so one day I can just add pyproject.toml and everything would continue to work without a single line of change to tox.ini).

[nicoddemus]

Having separate environments for build and testing is slower, but has the advantage that you don't risk having hard-coded paths in your wheel. If the build and testing environments are the same, you risk having hard coded paths and your build/tests will pass but your wheel will be in fact broken.

conda has two distinct steps, build and test, and for each one a new build environment is created. In fact, before the test environment is created and the test commands are executed, conda even renames the build environment to something else to make sure that the package doesn't have any hard coded paths on it.

[ionelmc]

So I've implemented wheel_build_env. I would like to see an example of the "hard-coded paths in wheel", cause right now I see wheel_build_env more of a way to quickly build universal wheels rather than something to prevent "hard-coded paths".

[ionelmc]

@gaborbernat when pyproject.toml handling was added in tox how come https://pypi.org/project/pep517/ wasn't used?

[gaborbernat]

Seemed too much for what it was needed for us, and we want to offer further functionalities (e.g. interpreter selection) all of which already is built into tox. That library offers from what I guaged more like isolated build (e.g. no focus on reusing the same interpreter etc).

[daa]

@ionelmc Split between build and test environments is important for me personally as a way to ensure proper specification of build and install requirements, for example it may be easy to forget to mention cffi as install requirement when it is mentioned as build requirement.

[ionelmc]

Just released 0.5.0.