search

storborg / python-packaging

Tutorial on how to structure Python packages
http://www.scotttorborg.com/python-packaging/
438 stars 133 forks source link

Top level tests directory #51

Open kousu opened 4 years ago

kousu commented 4 years ago

https://github.com/storborg/python-packaging/blob/35daf9938a63f7deb4b25cb1c5c65578c73f28e4/testing.rst recommends

tests [..] should be placed in a submodule of funniest. so that they can be imported, but won't pollute the global namespace.

There's a good case to be made that tests should not live inside your package, but instead inside a separate top-level tests/. This is how one of the PyPA core devs does it. This is also what https://packaging.python.org/tutorials/packaging-projects/ (which makes sense, it's written by the same group). The reason for this is it lets you cull tests from your binary distribution, since people installing from binary almost never care to run your tests or have them take up space -- though you need to go out of your way to make sure to re-include them in your sdist, but that's a one-line addition.

https://stackoverflow.com/questions/61151/where-do-the-python-unit-tests-go covers the three main options:

  1. Side-by-side your code (and says they prefer it; to be honest, so do I, it feels more right)
  2. In a top level tests/
  3. In a tests/ subpackage

The current recommendation in this doc is for 3. PyPA thinks you should use 2. What are your thoughts?

merwok commented 4 years ago

Side-by-side with the code only works if you are not using an src directory to isolate your package source (see https://blog.ganssle.io/articles/2019/08/test-as-installed.html)

Top-level tests is common. Depending on your build tool (setuptools or flit or poetry or other), you may need configuration to make sure that the tests are present in sdists, but not installed.

kousu commented 4 years ago

Side-by-side with the code only works if you are not using an src directory to isolate your package source (see https://blog.ganssle.io/articles/2019/08/test-as-installed.html)

Now I'm veering off-topic, but I wish there wasn't a distinction between installed/not installed. It isn't necessary in Javascript or Scheme or lots of other languages. I've been thinking about it and I think the reason comes down to the rule that submodules locate themselves -- and relative imports -- by __name__, splitting on "."s instead of by __file__, splitting on "/"s or better yet following "..". Can you think of any other reason?

Depending on your build tool (setuptools or flit or poetry or other), you may need configuration to make sure that the tests are present in sdists, but not installed.

Just to spread this knowledge around since it's going to get buried, PyPA now seems to be advocating either 

echo include tests/*.py >> MANIFEST.in

or using https://pypi.org/project/setuptools-scm/. But that seems weird to me; the manifest seems fine.