Sphinx-Action can't find target Python modules when custom sys.path(s) are specified in conf.py

[chamaoskurumi]

Tl;dr

When I create a sphinx doc locally it builds as expected, but sphinx-action creates an empty doc. It must have to do with the sphinx-action not finding the target python modules as specified in conf.py.

Any ideas to configure the sphinx-action or conf.py correctly?

Expected Sphinx Doc

When I build the sphinx doc locally on my machine via cd docs/ && make html the resulting html looks as expected

Empty Sphinx Doc generated by sphinx-action

My .github/workflows/sphinx_action.yml includes

    steps:
    # Checkout repo
    - uses: actions/checkout@v2
    # Build sphinx doc
    - uses: ammaraskar/sphinx-action@master
      with:
        docs-folder: "docs/"

and generates an empty skeleton of a Sphinx Doc

Project Setup

Project Structure

.
├── docs
│   ├── conf.py
│   ├── index.rst
│   ├── make.bat
│   ├── Makefile
│   ├── requirements.txt
│   └── sync.log
├── .github
│   └── workflows
│       └── sphinx_action.yml
├── .gitignore
├── mypackage
│   ├── __init__.py
│   ├── subfolder
│   │   ├── __init__.py
│   │   └── subclass.py
│   └── superclass.py
├── Pipfile
├── Pipfile.lock
└── README.md

conf.py Configuration

My docs/conf.py looks as follows. (Mind that I added three entries to the sys.path manually in order to make Sphinx find all python modules.)

# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
from m2r import MdInclude

sys.path.insert(0, os.path.abspath('..'))
sys.path.insert(0, os.path.abspath('../mypackage'))
sys.path.insert(0, os.path.abspath('../mypackage/subfolder'))

# -- Project information -----------------------------------------------------

project = 'MyPackage'
copyright = ''
author = 'Testuser'

# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc',
              'recommonmark'
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

#
source_suffix = ['.rst', '.md']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# document the init of a class, too
autoclass_content = 'both'

# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'nature'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

# from m2r to make `mdinclude` work
def setup(app):
    config = {
        # 'url_resolver': lambda url: github_doc_root + url,
        'auto_toc_tree_section': 'Contents',
        'enable_eval_rst': True,
    }

    # from m2r to make `mdinclude` work
    app.add_config_value('no_underscore_emphasis', False, 'env')
    app.add_config_value('m2r_parse_relative_links', False, 'env')
    app.add_config_value('m2r_anonymous_references', False, 'env')
    app.add_config_value('m2r_disable_inline_math', False, 'env')
    app.add_directive('mdinclude', MdInclude)

Pipfile

[[source]]
verify_ssl = true
url = "https://pypi.org/simple"
name = "pypi"

[dev-packages]

[packages]
m2r = {index = "pypi",version = "==0.2.1"}
pandas = {index = "pypi",version = "==1.0.3"}
sphinx = {index = "pypi",version = "==3.1.0"}
recommonmark = {index = "pypi",version = "==0.6.0"}
mypackage = "*"

[requires]
python_version = "3.8"

Note: I asked the same question on SO.

[ammaraskar]

Hey, that's really weird. Not sure why it's happening.

Could you show the logs for sphinx-action for when it generates those empty documentation files? Alternatively, you can also just push your project to Github so I can try recreating it myself (I tried looking through you repos but couldn't find it).

[ischoegl]

I initially ran into the same issue, but was able to resolve (and responded on SO). If you want to compare notes, here's the link to my repo .. it's very much work in progress, but the GitHub actions work.

PS: There are some differences here as I use setuptools to install my package, but the outcome was the same. In my case, I actually don't think that it's an issue with the action itself, as I have faint recollections of running into this earlier for sphinx documentation generated for a GitLab repo.

[adam-grant-hendry]

@chamaoskurumi Is the workflow running, passing, and not generating docs, or is the workflow failing? Can you share a link to your GitHub Actions run?

[chamaoskurumi]

Sorry @adam-grant-hendry , I cannot reproduce the error now as I don't have access anymore the code I used back then. Unfortunately I also can't remember if the workflow passed or not, but I'm pretty sure it didn't.

[adam-grant-hendry]

@chamaoskurumi No worries. Thanks for replying back! I agree with @ischoegl that your package was most likely not importable, which can cause this to happen. The sphinx.ext.autodoc docs state:

For Sphinx (actually, the Python interpreter that executes Sphinx) to find your module, it must be importable.

This can be done in one of two ways:

1. Make an editable install, i.e. pip install -e . (More on that here in the pip.pypa docs). This was the answer given by @ischoegl in your SO post

2. Manually add your package to sys.path in your conf.py files, which has been the traditional/"old-school" recommendation. The problem with this approach is if the interpreter can't find your module (i.e. a bad path is used) or it encounters an error when importing your package, then it still won't generate your docs. (That is, even if sys.path has a path, it might not be able to import that path.)

I generally don't recommend 2, but if you choose it, you can safeguard against this from happening again by using importlib.import_module in a try...except clause:

from importlib import import_module

package_name = 'my_package'  # Replace with your package name

try:
    import_module(package_name)
except ImportError as err:
    raise ImportError(f'Package {package_name} could not be imported.')

It's also generally considered best practice nowadays to use the pathlib module instead of os.path, especially when mucking with sys.path. In fact, pathlib.Path behaves similarly to sphinx.path: both treat paths as objects instead of strings (more on why that makes sense in this realpython.com article).

The second recommendation in your SO post to use __file__ to get an absolute path to your package is also accurate, but there are some caveats. Barry Warsaw gave a great PyCon talk on this: I highly recommend watching it to everyone I talk to.

Ultimately, the issue comes down to your package being importable, so I recommend 1 like @ischoegl did. It's much simpler and cleaner all around. Just do import mypackage and Python will automatically error if it can't import it.

[adam-grant-hendry]

@ammaraskar @chamaoskurumi Can this issue be closed?