`sphinx.ext.autosummary` does not process a subpackage, even though it imports without errors

[JesseTG]

Describe the bug

I'm trying to generate API documentation from this repository using sphinx.ext.autosummary. However, the contents of libretro.driver are not being processed.

Here are some more details:

• Each package has an __init__.py that consists solely of from .subpackage import * statements.
• All modules list their public symbols in __all__.
• Sphinx does not emit any errors, as far as I can tell.
• The package's entire public API is exposed through the libretro namespace, and calling from libretro import * imports everything successfully.
• Here's a screenshot illustrating the issue:

How to Reproduce

1. Clone the aforementioned repository
2. Check out the jtg/sphinx-bug branch
3. Set up a venv with the Sphinx and sphinx-autobuild packages installed
4. Activate the venv and run this command: sphinx-autobuild -j auto --ignore "./docs/libretro/*" --keep-going -T docs build/docs
5. Open the docs in-browser and look for documentation for a class named ArrayAudioDriver. You won't find it.

conf.py is defined here, and index.rst is defined here.

Environment Information

Platform:              win32; (Windows-10-10.0.19045-SP0)
Python version:        3.12.3 (tags/v3.12.3:f6650f9, Apr  9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)])
Python implementation: CPython
Sphinx version:        7.2.6
Docutils version:      0.20.1
Jinja2 version:        3.1.3
Pygments version:      2.17.2

Sphinx extensions

extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.autosummary',
]

The most recent commit in my repo has sphinx.ext.doctest and sphinx.ext.duration, but the issue occurs even when those are excluded.

[electric-coder]

I can't understand from the description and screenshots what the problem is. Aren't libretro.api and libretro.driver on the same level of depth and aren't both being included in the toctree? Within that structure what exactly is missing?

Do you mean that you were to have 2 children (libretro.api and libretro.driver) within the top level package libretro and the subpackages are being listed with incorrect nestings and levels of depth? What seems to be happening is that you have docs for individual modules at the same level as docs for packages. There's nothing wrong with that. You also make extensive use of __all__ but in the base package's __init__.py you only include the subpackages not the modules inside the top-level package. I'm unsure how autosummary would behave in that case.

[JesseTG]

I can't understand from the description and screenshots what the problem is. Aren't libretro.api and libretro.driver on the same level of depth and aren't both being included in the toctree?

Yes, that's correct.

Within that structure what exactly is missing?

libretro.driver should have doc pages for the submodules that it contains. It doesn't; they're not even generated.

Do you mean that you were to have 2 children (libretro.api and libretro.driver) within the top level package libretro and the subpackages are being listed with incorrect nestings and levels of depth?

No, libretro.driver is the only offender. All other packages are correctly nested (e.g. libretro.core is a single module not nested in api or driver).

What seems to be happening is that you have docs for individual modules at the same level as docs for packages. There's nothing wrong with that.

I agree, that's not the issue I'm facing.

You also make extensive use of __all__ but in the base package's __init__.py you only include the subpackages not the modules inside the top-level package.

You mean this base package? I'm not sure what you mean here, could you elaborate?

[JesseTG]

I don't believe this...renaming the offending package from libretro.driver to libretro.impl works around the issue!

Is the string driver special at all?

[JesseTG]

Using the name libretro.drivers also works as expected.

[JesseTG]

@thegamecracks was looking into this and offered this insight: