Proposal: switch to src layout instead of flat/ad-hoc layout

[namcsi]

I'd like to switch the layout of the project template over to the src layout. The two types of layouts I'm referencing are these.

The reason I'd like this change is that, as hinted at in the breakdown in the link above, the flat layout can lead to some subtle error when testing or configuring package distribution. This is due to the fact that Python interpreter includes the current working directory as the first item on the import path, so e.g. when running tests they get run against the current state of the repo, and not against what a user would get if they installed the package using pip. These links have some further examples: link1 link2.

To appeal to a higher authority, the pypa packaging tutorial also uses the src layout due to these points;)

I'd be happy to open a pull request if we agree that this is something we'd like to do. We could also include some instructions on how to create an editable install and how to build a project for beginners to reduce any possible friction. Let me know your thoughts @rkaminsk @susuhahnml!

[rkaminsk]

I feel that the advantages are slim but I am also not opposed to this change. Please provide a PR, if you want to have this. Just make sure to test everything. Both nox and precommit setup have to work.

[rkaminsk]

I tried to encourage people to use nox, which takes care of setting up environments. The package is even installed in editable mode into these environments. However, an editor might need an environment for auto-completion, linting, etc. It is possible to reuse the nox environments. For example, to run the tests:

source .nox/test/bin/activate
python -m unittest discover

Of course, one can also setup ones own environment. There is even a dev variant to install tools:

python -m venv dev
source dev/bin/activate
pip install -e '.[dev]'
python -m unittest discover

[namcsi]

Yep, I'm planning on using nox in my project:) But the issue with testing mentioned above also applies when running tests through nox. For example, in the template we have the nox session:

@nox.session(python=PYTHON_VERSIONS)
def test(session):
    session.install("-e", ".[test]")
    session.run("coverage", "run", "-m", "unittest", "discover", "-v")
    session.run("coverage", "report", "-m", "--fail-under=100")

Even though nox installed the package (along with test dependencies) into a virtual environment to test it, with the flat layout, when the test commands are run, the imports in e.g. test_main.py:

from fillname.utils.logger import setup_logger
from fillname.utils.parser import get_parser

will be coming from the source tree and not from the installed version of the package nox set up in a venv for us to test. This is again due to the fact that the current working directory is added as the first entry into the python import path, thus having priority over the virtualenv.

So this is why I'd prefer using the src layout.

[namcsi]

I'll put together a PR soon then so we have something concrete.

[rkaminsk]

will be coming from the source tree and not from the installed version of the package nox set up in a venv for us to test. This is again due to the fact that the current working directory is added as the first entry into the python import path, thus having priority over the virtualenv.

Yes, but the package installation is just a symlink anyway. I can imagine that there can be some issues in more complex setups. In any case, we can switch to the recommended source layout.

[namcsi]

Done! Let me know if you see anything off @rkaminsk.