Package not honoring sphinxcontrib namespace package structure

[devmonkey22]

Should the autodoc_pydantic sphinxcontrib/__init__.py file file exist? Or does it need to declare the namespace like sphinxcontrib.apidoc does? According to PyPA Packaging namespace packages, it shouldn't exist.

When the __init__.py file is present and the autodoc_pydantic extension is registered (using Sphinx 5.3.0), I get an error:

Extension error:
Could not import extension sphinxcontrib.autodoc_pydantic (exception: No module named 'sphinxcontrib.autodoc_pydantic')

If I manually remove the __init__.py file then run sphinx again, it seems to run fine.

Looking at other extensions, they don't define a sphinxcontrib/__init__.py file like:

• https://github.com/sphinx-contrib/autofile/tree/main/src/sphinxcontrib/
• https://github.com/sphinx-contrib/confluencebuilder/tree/main/sphinxcontrib
• https://github.com/sphinx-contrib/spelling/tree/main/sphinxcontrib

From what I can see, apidoc may be the only one declaring the namespace?

[devmonkey22]

Looking further, based on Sphinx Contrib's cookiecutter project, seems like they want to use an __init__.py but declare the namespace in it. However, this approach didn't seems to fix the issue in my environment?

[devmonkey22]

Also, after I set up the repository locally on my Ubuntu 22.04 and Python 3.10.12 VM with poetry (after sudo apt-get install graphviz graphviz-dev too), then ran poetry run pytest, I get the same errors in the tests:

I'll work to submit a small PR and see how things go.

[daquinteroflex]

Having the same issue now trying to set this up for development, wonder if this is why I am having issues getting the autodoc_pydnatic functionality to build on my project

[daquinteroflex]

Maybe related so posting this here:

(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ make clean && make html
Running Sphinx v5.3.0

Extension error:
Could not import extension sphinx_rtd_theme (exception: No module named 'sphinx_rtd_theme')
make: *** [Makefile:20: html] Error 2
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ exit
exit
daquintero@dxps:~/flexcompute/autodoc_pydantic$ poetry install -E dev
Installing dependencies from lock file

Package operations: 20 installs, 0 updates, 0 removals

  • Installing distlib (0.3.7)
  • Installing filelock (3.12.2)
  • Installing platformdirs (4.0.0)
  • Installing iniconfig (2.0.0)
  • Installing mccabe (0.6.1)
  • Installing pluggy (1.2.0)
  • Installing py (1.11.0)
  • Installing pycodestyle (2.7.0)
  • Installing pyflakes (2.3.1)
  • Installing six (1.16.0)
  • Installing sphinxcontrib-jquery (4.1)
  • Installing virtualenv (20.24.7)
  • Installing coverage (7.2.7)
  • Installing flake8 (3.9.2)
  • Installing pytest (7.4.3)
  • Installing sphinx-copybutton (0.4.0)
  • Installing sphinx-rtd-theme (1.3.0)
  • Installing sphinx-tabs (3.4.4)
  • Installing sphinxcontrib-mermaid (0.7.1)
  • Installing tox (3.28.0)

Installing the current project: autodoc_pydantic (2.0.1)
daquintero@dxps:~/flexcompute/autodoc_pydantic$ poetry shell
Spawning shell within /home/daquintero/.cache/pypoetry/virtualenvs/autodoc-pydantic-QOa8Dwz7-py3.11
gpg-agent[51818]: option file '/home/daquintero/.gnupg/gpg-agent.conf': No such file or directory
daquintero@dxps:~/flexcompute/autodoc_pydantic$ . /home/daquintero/.cache/pypoetry/virtualenvs/autodoc-pydantic-QOa8Dwz7-py3.11/bin/activate
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ ls
changelog.rst  docs  LICENSE  poetry.lock  pyproject.toml  README.md  sphinxcontrib  tests  tox.ini
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ cd docs/
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ ls
make.bat  Makefile  source
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ make clean && make html
Running Sphinx v5.3.0

Extension error:
Could not import extension sphinxcontrib.autodoc_pydantic (exception: No module named 'sphinxcontrib.autodoc_pydantic')
make: *** [Makefile:20: html] Error 2
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ cd
(autodoc-pydantic-py3.11) daquintero@dxps:~$ 
(autodoc-pydantic-py3.11) daquintero@dxps:~$ ls
Desktop  Documents  Downloads  flexcompute  java_error_in_pycharm_.hprof  legacy  Music  phd  Pictures  Public  snap  someconfig.txt  Templates  Videos
(autodoc-pydantic-py3.11) daquintero@dxps:~$ cd flexcompute/
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute$ ls
autodoc_pydantic  tidy3d  tidy3d-notebooks
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute$ cd autodoc_pydantic/
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ ls
changelog.rst  docs  LICENSE  poetry.lock  pyproject.toml  README.md  sphinxcontrib  tests  tox.ini
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ make
make: *** No targets specified and no makefile found.  Stop.
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ sphinx
sphinx: command not found
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ exit
exit
daquintero@dxps:~/flexcompute/autodoc_pydantic$ cd docs/
daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ poetry run make html
Running Sphinx v5.3.0

Extension error:
Could not import extension sphinxcontrib.autodoc_pydantic (exception: No module named 'sphinxcontrib.autodoc_pydantic')
make: *** [Makefile:20: html] Error 2
daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ code .
daquintero@dxps:~/flexcompute/autodoc_pydantic/docs$ cd ..
daquintero@dxps:~/flexcompute/autodoc_pydantic$ ls
changelog.rst  docs  LICENSE  poetry.lock  pyproject.toml  README.md  sphinxcontrib  tests  tox.ini
daquintero@dxps:~/flexcompute/autodoc_pydantic$ cat docs/make.bat 
@ECHO OFF

pushd %~dp0

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
        set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build

if "%1" == "" goto help

%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
        echo.
        echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
        echo.installed, then set the SPHINXBUILD environment variable to point
        echo.to the full path of the 'sphinx-build' executable. Alternatively you
        echo.may add the Sphinx directory to PATH.
        echo.
        echo.If you don't have Sphinx installed, grab it from
        echo.http://sphinx-doc.org/
        exit /b 1
)

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%

:end
popd
daquintero@dxps:~/flexcompute/autodoc_pydantic$ cat docs/Makefile 
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS    ?=
SPHINXBUILD   ?= sphinx-build
SOURCEDIR     = source
BUILDDIR      = build

# Put it first so that "make" without argument is like "make help".
help:
        @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
        @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
daquintero@dxps:~/flexcompute/autodoc_pydantic$ poetry run sphinx docs/source/ docs/build
Command not found: sphinx
daquintero@dxps:~/flexcompute/autodoc_pydantic$ poetry shell
Spawning shell within /home/daquintero/.cache/pypoetry/virtualenvs/autodoc-pydantic-QOa8Dwz7-py3.11
gpg-agent[52760]: option file '/home/daquintero/.gnupg/gpg-agent.conf': No such file or directory
daquintero@dxps:~/flexcompute/autodoc_pydantic$ . /home/daquintero/.cache/pypoetry/virtualenvs/autodoc-pydantic-QOa8Dwz7-py3.11/bin/activate
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ pip list
Package                       Version    Editable project location
----------------------------- ---------- ---------------------------------------------
alabaster                     0.7.13
annotated-types               0.5.0
autodoc_pydantic              2.0.1      /home/daquintero/flexcompute/autodoc_pydantic
Babel                         2.13.1
certifi                       2023.11.17
charset-normalizer            3.3.2
coverage                      7.2.7
distlib                       0.3.7
docutils                      0.18.1
filelock                      3.12.2
flake8                        3.9.2
idna                          3.4
imagesize                     1.4.1
iniconfig                     2.0.0
Jinja2                        3.1.2
MarkupSafe                    2.1.3
mccabe                        0.6.1
packaging                     23.2
pip                           23.3.1
platformdirs                  4.0.0
pluggy                        1.2.0
py                            1.11.0
pycodestyle                   2.7.0
pydantic                      2.5.1
pydantic_core                 2.14.3
pydantic-settings             2.0.3
pyflakes                      2.3.1
Pygments                      2.17.2
pytest                        7.4.3
python-dotenv                 0.21.1
requests                      2.31.0
setuptools                    68.2.2
six                           1.16.0
snowballstemmer               2.2.0
Sphinx                        5.3.0
sphinx-copybutton             0.4.0
sphinx-rtd-theme              1.3.0
sphinx-tabs                   3.4.4
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-mermaid         0.7.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
tox                           3.28.0
typing_extensions             4.7.1
urllib3                       2.0.7
virtualenv                    20.24.7
(autodoc-pydantic-py3.11) daquintero@dxps:~/flexcompute/autodoc_pydantic$ exit
exit
daquintero@dxps:~/flexcompute/autodoc_pydantic$ poetry run Sphinx docs/source/ docs/build
Command not found: Sphinx
daquintero@dxps:~/flexcompute/autodoc_pydantic$ poetry run python3 -m sphinx docs/source/ docs/build
Running Sphinx v5.3.0

Extension error:
Could not import extension sphinxcontrib.autodoc_pydantic (exception: No module named 'sphinxcontrib.autodoc_pydantic')

[mansenfranzen]

@devmonkey22 and @daquinteroflex thanks a lot for looking into this issue and providing a PR. I agree with your observation that __init__.py should no longer be present in the sphinxcontrib namespace package. In fact, I used to have issues running tests in some environments due to failed imports, too. But I wasn't aware that this could be caused by the somewhat incorrect/inconsistent usage of namespace packages.

I will take a closer look at it. While removing the init file, we need to be sure that we don't cause unrecognized backwards-incompatible breaking changes.

[mansenfranzen]

@all-contributors please add @devmonkey22 for code, bug and test

[allcontributors[bot]]

@mansenfranzen

I've put up a pull request to add @devmonkey22! :tada:

[mansenfranzen]

@all-contributors please add @daquinteroflex for review

[allcontributors[bot]]

@mansenfranzen

I've put up a pull request to add @daquinteroflex! :tada:

[mansenfranzen]

Before merging the related PR, it would be great if you could test the changed behavior. To do so, please install the current dev release in your doc-building-environment via pip install git+https://github.com/mansenfranzen/autodoc_pydantic.git@v2.1.0-a.1 and rebuild your docs.

Please feel free to reopen if any problem remains. I will release the new version upcoming Monday.