Installation can't seen src (correction: src not added to path)

[LydiaFrance]

The following breaks:

my_package
- examples/
-- notebook_walkthrough.ipynb
- src/
-- my_package/ 
--- __init__.py
--- myfunctions.py

This fails inside the notebook:

from my_package import function

{ "ename": "ModuleNotFoundError", "evalue": "No module named 'my_package'", "output_type": "error",

Editing the setup.cfg fixes this:

[metadata]
name = my_package
version = 0.1.0

[options]
package_dir =
    = src
packages = find:

[options.packages.find]
where = src

[options.extras_require]
test =
    pytest >=6
    pytest-cov >=3
    coloredlogs

Also hardcoding fixes it: sys.path.append('../../src/')

[phinate]

Just double-checking this is the right repo to leave this comment? This is a pyproject.toml workflow, and shouldnt contain a setup.cfg.

[phinate]

Realising that probably isn't that useful of a comment! The way to fix this problem should just be to include a pyproject.toml, which I can't see in your directory tree, which is why I questioned if you generated this project with this template, as that should come batteries-included!

If you do have a pyproject.toml, there might have been an installation step missing:

pip install -e .

which is necessary for src/ layouts, as they will not automatically get picked up by Python when it searches for modules.

Here are a couple of links that I use to remind myself of these things:

• the PyPA guide on packaging projects
• Python docs on src/ vs flat layouts

Hope this helps!

[LydiaFrance]

@phinate Ah sorry -- i do have a pyproject.toml and have definitely done pip install -e . It's exactly using your template.

[LydiaFrance]

I've just made a minimal example and recreated it again, this time no setup.cfg.

https://github.com/LydiaFrance/birdlabelling

python -m venv .venv source .venv/bin/activate pip install --upgrade pip pip install copier python -m copier copy gh:alan-turing-institute/python-project-template birdlabelling <- for the record, copier alone doesn't work, it doesn't look inside the environment and can't load pyyaml pip install -e .

Here's my files:

.
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── directory-structure.md
├── examples
│   └── walkthrough.ipynb
├── pyproject.toml
└── src
    ├── birdlabelling
    │   ├── __init__.py
    │   ├── __pycache__
    │   │   ├── __init__.cpython-312.pyc
    │   │   └── labelling.cpython-312.pyc
    │   └── labelling.py

Within pyproject.toml:

[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"

[project]
name = "birdlabelling"
version = "0.1.0"
authors = [
  { name = "Lydia France", email = " " },
]
description = "Trying out minimal example"
readme = "README.md"
requires-python = ">=3.10"
classifiers = [
  "Development Status :: 1 - Planning",
  "Intended Audience :: Science/Research",
  "Intended Audience :: Developers",
  "License :: OSI Approved :: MIT License",
  "Operating System :: OS Independent",
  "Programming Language :: Python",
  "Programming Language :: Python :: 3",
  "Programming Language :: Python :: 3 :: Only",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: 3.12",
  "Topic :: Scientific/Engineering",
]
dependencies = []

[project.optional-dependencies]
dev = [
  "pytest >=6",
  "pytest-cov >=3",
  "pre-commit",
]

[project.urls]
Homepage = "https://github.com/LydiaFrance/birdlabelling"
"Bug Tracker" = "https://github.com/LydiaFrance/birdlabelling/issues"
Discussions = "https://github.com/LydiaFrance/birdlabelling/discussions"
Changelog = "https://github.com/LydiaFrance/birdlabelling/releases"

[tool.pytest.ini_options]
minversion = "6.0"
addopts = ["-ra", "--showlocals", "--strict-markers", "--strict-config"]
xfail_strict = true
filterwarnings = [
  "error",
]
log_cli_level = "INFO"
testpaths = [
  "tests",
]

[tool.coverage]
run.source = ["birdlabelling"]
port.exclude_lines = [
  'pragma: no cover',
  '\.\.\.',
  'if typing.TYPE_CHECKING:',
]

[tool.ruff]
src = ["src"]
exclude = []
line-length = 88  # how long you want lines to be

[tool.ruff.format]
docstring-code-format = true  # code snippets in docstrings will be formatted

[tool.ruff.lint]
select = [
  "E", "F", "W", # flake8
  "B",           # flake8-bugbear
  "I",           # isort
  "ARG",         # flake8-unused-arguments
  "C4",          # flake8-comprehensions
  "EM",          # flake8-errmsg
  "ICN",         # flake8-import-conventions
  "ISC",         # flake8-implicit-str-concat
  "G",           # flake8-logging-format
  "PGH",         # pygrep-hooks
  "PIE",         # flake8-pie
  "PL",          # pylint
  "PT",          # flake8-pytest-style
  "RET",         # flake8-return
  "RUF",         # Ruff-specific
  "SIM",         # flake8-simplify
  "UP",          # pyupgrade
  "YTT",         # flake8-2020
  "EXE",         # flake8-executable
]
ignore = [
  "PLR",    # Design related pylint codes
  "ISC001", # Conflicts with formatter
]
unfixable = [
  "F401",   # Would remove unused imports
  "F841",   # Would remove unused variables
]
flake8-unused-arguments.ignore-variadic-names = true  # allow unused *args/**kwargs

Within the notebook examples/walkthrough.ipynb this fails:

from birdlabelling import myfunction

Fails

birdlabelling module not found

This fixes it:

import sys

sys.path.append('../src/')

from birdlabelling import myfunction

[phinate]

I can't reproduce this error with those steps.

My suspicion is that the Python env in your notebook does not match the Python in your terminal. You can test this by running

import sys

print(sys.executable)

in your notebook, and see if that matches

which python

in the terminal.

[LydiaFrance]

Okay, after some digging.. the environment was in the right place, python was in the right place, executable path was in the right place, the notebook had the right kernel.... but pip was coming from homebrew. 🤦‍♀️ Thanks!

[phinate]

I see! I've gotten a little lazy with this, but one should technically always prefer running python -m pip ... in all cases to avoid this :)

[phinate]

Nice detective work!