readthedocs / readthedocs.org

The source code that powers readthedocs.org
https://readthedocs.org/
MIT License
8.04k stars 3.58k forks source link

Docs build locally but fail when running on readthedocs.io due to python import errors #8083

Closed Tasty213 closed 3 years ago

Tasty213 commented 3 years ago

Details

Read the docs URL: https://github.com/Reclaim-The-Night-Leeds/crime-hotspots-uk/tree/docs Github source: https://crime-hotspots-uk.readthedocs.io/en/docs/ Read the docs username: rtnleeds

Expected Result

Docs to build on readthedocs.io in the same way as If I run tox -e docs on local computer

Actual Result

Read the docs complains with the following error

WARNING: autodoc: failed to import module 'constants' from module 'crime_hotspots_uk'; the following exception was raised:
No module named 'crime_hotspots_uk'
WARNING: autodoc: failed to import module 'dataCollection' from module 'crime_hotspots_uk'; the following exception was raised:
No module named 'crime_hotspots_uk'
WARNING: autodoc: failed to import module 'crime_hotspots_uk'; the following exception was raised:
No module named 'crime_hotspots_uk'

Additional info

As the issue is only when building the docs remotely i keep have to commit my local changes to the repo without being able to test them is there a way to test locally?

Tasty213 commented 3 years ago

.readthedocs.yaml` file

# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Build documentation in the docs/ directory with Sphinx
sphinx:
  configuration: docs/conf.py

# Build documentation with MkDocs
#mkdocs:
#  configuration: mkdocs.yml

# Optionally build your docs in additional formats such as PDF
formats:
  - pdf

python:
  version: 3.8
  install:
    - requirements: docs/requirements.txt

conf.py file

# This file is execfile()d with the current directory set to its containing dir.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
#
# All configuration values have a default; values that are commented out
# serve to show the default.

import os
import sys
import inspect
import shutil

# -- Path setup --------------------------------------------------------------

__location__ = os.path.join(
    os.getcwd(), os.path.dirname(inspect.getfile(inspect.currentframe()))
)

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.join(__location__, "../src/crime-hotspots-uk/"))

# -- Run sphinx-apidoc -------------------------------------------------------
# This hack is necessary since RTD does not issue `sphinx-apidoc` before running
# `sphinx-build -b html . _build/html`. See Issue:
# https://github.com/rtfd/readthedocs.org/issues/1139
# DON'T FORGET: Check the box "Install your project inside a virtualenv using
# setup.py install" in the RTD Advanced Settings.
# Additionally it helps us to avoid running apidoc manually

try:  # for Sphinx >= 1.7
    from sphinx.ext import apidoc
except ImportError:
    from sphinx import apidoc

print('*' * 10)
print('__location__', __location__)
print('*' * 10)

output_dir = os.path.join(__location__, "api")
module_dir = os.path.join(__location__, "../src/crime_hotspots_uk/")
try:
    shutil.rmtree(output_dir)
except FileNotFoundError:
    pass

try:
    import sphinx

    cmd_line_template = (
        "sphinx-apidoc --implicit-namespaces -f -o {outputdir} {moduledir}"
    )
    cmd_line = cmd_line_template.format(outputdir=output_dir, moduledir=module_dir)

    args = cmd_line.split(" ")
    if tuple(sphinx.__version__.split(".")) >= ("1", "7"):
        # This is a rudimentary parse_version to avoid external dependencies
        args = args[1:]

    apidoc.main(args)
except Exception as e:
    print("Running `sphinx-apidoc` failed!\n{}".format(e))

# -- General configuration ---------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
# needs_sphinx = '1.0'

# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.intersphinx",
    "sphinx.ext.todo",
    "sphinx.ext.autosummary",
    "sphinx.ext.viewcode",
    "sphinx.ext.coverage",
    "sphinx.ext.doctest",
    "sphinx.ext.ifconfig",
    "sphinx.ext.mathjax",
    "sphinx.ext.napoleon",
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]

# Configure AutoStructify
# https://recommonmark.readthedocs.io/en/latest/auto_structify.html
def setup(app):
    from recommonmark.transform import AutoStructify

    params = {
        "enable_auto_toc_tree": True,
        "auto_toc_tree_section": "Contents",
        "auto_toc_maxdepth": 2,
        "enable_eval_rst": True,
        "enable_math": True,
        "enable_inline_math": True,
    }
    app.add_config_value("recommonmark_config", params, True)
    app.add_transform(AutoStructify)
    app.connect("autodoc-skip-member", skip)

# Ensure that __init__ functions are always documented
def skip(app, what, name, obj, would_skip, options):
    if name == "__init__":
        return False
    return would_skip

# Enable markdown
extensions.append("recommonmark")

# The suffix of source filenames.
source_suffix = [".rst", ".md"]

# The encoding of source files.
# source_encoding = 'utf-8-sig'

# The master toctree document.
master_doc = "index"

# General information about the project.
project = "crime-hotspots-uk"
copyright = "2021, Reclaim The Night Leeds"

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = ""  # Is set by calling `setup.py docs`
# The full version, including alpha/beta/rc tags.
release = ""  # Is set by calling `setup.py docs`

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
# language = None

# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
# today = ''
# Else, today_fmt is used as the format for a strftime call.
# today_fmt = '%B %d, %Y'

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", ".venv"]

# The reST default role (used for this markup: `text`) to use for all documents.
# default_role = None

# If true, '()' will be appended to :func: etc. cross-reference text.
# add_function_parentheses = True

# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
# add_module_names = True

# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
# show_authors = False

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"

# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []

# If true, keep warnings as "system message" paragraphs in the built documents.
# keep_warnings = False

# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
html_theme = "sphinx_rtd_theme"

# Theme options are theme-specific and customize the look and feel of a theme
# further.  For a list of options available for each theme, see the
# documentation.
html_theme_options = {}

# Add any paths that contain custom themes here, relative to this directory.
# html_theme_path = []

# The name for this set of Sphinx documents.  If None, it defaults to
# "<project> v<release> documentation".
try:
    from crime_hotspots_uk import __version__ as version
except ImportError:
    pass
else:
    release = version

# A shorter title for the navigation bar.  Default is the same as html_title.
html_short_title = 'Crime Hotspots UK Docs'

# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
# html_logo = ""

# The name of an image file (within the static path) to use as favicon of the
# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
# html_favicon = None

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]

# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
# html_last_updated_fmt = '%b %d, %Y'

# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
# html_use_smartypants = True

# Custom sidebar templates, maps document names to template names.
# html_sidebars = {}

# Additional templates that should be rendered to pages, maps page names to
# template names.
# html_additional_pages = {}

# If false, no module index is generated.
# html_domain_indices = True

# If false, no index is generated.
# html_use_index = True

# If true, the index is split into individual pages for each letter.
# html_split_index = False

# If true, links to the reST sources are added to the pages.
# html_show_sourcelink = True

# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
# html_show_sphinx = True

# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
# html_show_copyright = True

# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it.  The value of this option must be the
# base URL from which the finished HTML is served.
# html_use_opensearch = ''

# This is the file name suffix for HTML files (e.g. ".xhtml").
# html_file_suffix = None

# Output file base name for HTML help builder.
htmlhelp_basename = "crime-hotspots-uk-doc"

# -- Options for LaTeX output ------------------------------------------------

latex_elements = {
    # The paper size ("letterpaper" or "a4paper").
    # "papersize": "letterpaper",
    # The font size ("10pt", "11pt" or "12pt").
    # "pointsize": "10pt",
    # Additional stuff for the LaTeX preamble.
    # "preamble": "",
}

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
    ("index", "user_guide.tex", "crime-hotspots-uk Documentation", "George Sykes", "manual")
]

# The name of an image file (relative to this directory) to place at the top of
# the title page.
# latex_logo = ""

# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
# latex_use_parts = False

# If true, show page references after internal links.
# latex_show_pagerefs = False

# If true, show URL addresses after external links.
# latex_show_urls = False

# Documents to append as an appendix to all manuals.
# latex_appendices = []

# If false, no module index is generated.
# latex_domain_indices = True

# -- External mapping --------------------------------------------------------
python_version = ".".join(map(str, sys.version_info[0:2]))
intersphinx_mapping = {
    "sphinx": ("http://www.sphinx-doc.org/en/stable", None),
    "python": ("https://docs.python.org/" + python_version, None),
    "matplotlib": ("https://matplotlib.org", None),
    "numpy": ("https://docs.scipy.org/doc/numpy", None),
    "sklearn": ("https://scikit-learn.org/stable", None),
    "pandas": ("https://pandas.pydata.org/pandas-docs/stable", None),
    "scipy": ("https://docs.scipy.org/doc/scipy/reference", None),
    "pyscaffold": ("https://pyscaffold.org/en/stable", None),
}

tox.ini file:

# Tox configuration file
# Read more under https://tox.readthedocs.org/
# THIS SCRIPT IS SUPPOSED TO BE AN EXAMPLE. MODIFY IT ACCORDING TO YOUR NEEDS!

[tox]
minversion = 3.15
envlist = default

[testenv]
description = invoke pytest to run automated tests
isolated_build = True
setenv =
    TOXINIDIR = {toxinidir}
passenv =
    HOME
extras =
    testing
commands =
    pytest {posargs}
deps = nbformat
    nbconvert
    ipykernel
    ipywidgets

[testenv:{clean,build}]
description =
    Build (or clean) the package in isolation according to instructions in:
    https://setuptools.readthedocs.io/en/latest/build_meta.html#how-to-use-it
    https://github.com/pypa/pep517/issues/91
    https://github.com/pypa/build
# NOTE: build is still experimental, please refer to the links for updates/issues
skip_install = True
changedir = {toxinidir}
deps =
    build: build[virtualenv]
commands =
    clean: python -c 'from shutil import rmtree; rmtree("build", True); rmtree("dist", True)'
    build: python -m build .
# By default `build` produces wheels, you can also explicitly use the flags `--sdist` and `--wheel`

[testenv:{docs,doctests}]
description = invoke sphinx-build to build the docs/run doctests
setenv =
    DOCSDIR = {toxinidir}/docs
    BUILDDIR = {toxinidir}/docs/_build
    docs: BUILD = html
    doctests: BUILD = doctest
deps =
    -r {toxinidir}/docs/requirements.txt
    # ^  requirements.txt shared with Read The Docs
commands =
    sphinx-build -b {env:BUILD} -d "{env:BUILDDIR}/doctrees" "{env:DOCSDIR}" "{env:BUILDDIR}/{env:BUILD}" {posargs}

[testenv:publish]
description =
    Publish the package you have been developing to a package index server.
    By default, it uses testpypi. If you really want to publish your package
    to be publicly accessible in PyPI, use the `-- --repository pypi` option.
skip_install = True
changedir = {toxinidir}
passenv =
    DISPLAY
    WAYLAND_DISPLAY
    DBUS_SESSION_BUS_ADDRESS
deps = twine
    dbus-python
    SecretStorage
    keyring
commands =
    python -m twine check dist/*
    python -m twine upload {posargs:--repository testpypi} dist/*
Tasty213 commented 3 years ago

Don't know if this is relevant but my docs file are in markdown format

stsewd commented 3 years ago

@Tasty213 hi, I don't see a .readthedocs.yaml file in https://github.com/Reclaim-The-Night-Leeds/crime-hotspots-uk/tree/docs, also, instead of https://github.com/Reclaim-The-Night-Leeds/crime-hotspots-uk/blob/5ea9f4ab0bbc9dd76d840bd0837977037f163090/docs/conf.py#L17

you should use something like

from pathlib import Path

here = Path(__file__).parent

that way it doesn't depend on the cwd the script was called from.

Tasty213 commented 3 years ago

Thanks for the prompt reply.

I've pushed the suggested edits (changed path method and added yaml)

my yaml is:

# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Build documentation in the docs/ directory with Sphinx
sphinx:
  configuration: docs/conf.py

# Build documentation with MkDocs
#mkdocs:
#  configuration: mkdocs.yml

# Optionally build your docs in additional formats such as PDF
formats:
  - pdf

python:
  version: 3.8
  install:
    - requirements: docs/requirements.txt
stsewd commented 3 years ago

@Tasty213 looks like you need one level up when injecting the path

https://github.com/Reclaim-The-Night-Leeds/crime-hotspots-uk/blob/f3ed874660241374d7ee037200f5b7072911554e/docs/conf.py#L23

That's

sys.path.insert(0, os.path.join(__location__, "../src/"))

since apidoc is trying to use the package as crime_hotspots_uk.dataCollection, but you may added it in addition to the other one in case you are making use of it in another way.

Tasty213 commented 3 years ago

Thanks alot works now!