Document class inheriting from LightningModule

[gunthergl]

❓

What is your question?

Given the code, I would assume that using sphinx, all 3 classes are documented. In fact the class inheriting from LightningModule is not documented. I post this as a how-to and hope for help because that could be a plain problem on my side, otherwise it would be a bug.

Using sphinx-apidoc, relevant generated part:

.. automodule:: ccc.models.exampleClass
   :members:
   :undoc-members:

Code

# ccc/models/exampleClass
from pytorch_lightning import LightningModule

class classA(object):
    def __init__(self, a: int):
        """
        Args:
            a:
                asdf
        """
        pass

class classB(LightningModule):
    def __init__(self, a: int):
        """
        Args:
            a:
                asdf
        """
        pass

class classC(classA):
    def __init__(self, a: int):
        """
        Args:
            a:
                asdf
        """
        pass

What's your environment?

• OS: [e.g. iOS, Linux, Win] Windows + WSL
• Packaging [e.g. pip, conda] poetry (pip)
• Version [e.g. 0.5.2.1] 0.8.5

[awaelchli]

I couldn't yet figure out what the problem is. Do you also get warnings/error messages when you generate html?

[gunthergl]

I don't. But do you also have the problem? That would already calm me a bit.

[awaelchli]

I just inserted your example class in a random place in the Lightning docs with automodule/autoclass, because I was too lazy to configure sphinx in a new folder. Don't know sphinx very well :)

But do you also have the problem?

Yes, the html didn't show the subclass of LightningModule

Perhaps you could share your config for sphinx? That may help.

[gunthergl]

Yes, I also do no really know sphinx - I am happy as long as it works. I just extended sphinx's quickstart configuration.

# -*- coding: utf-8 -*-
#
# Configuration file for the Sphinx documentation builder.
#
# This file does only contain a selection of the most common options. For a
# full list see the documentation:
# http://www.sphinx-doc.org/en/master/config

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
sys.path.insert(0, os.path.abspath('../../ccc/conf/schema/'))
sys.path.insert(0, os.path.abspath('../../'))
autodoc_mock_imports = [
    "torch", "torch_geometric", "flowio", "numpy", "matplotlib",
    "hydra", "pandas", "flowkit", "yaml", "omegaconf", "pytorch_lightning"]
autodoc_default_options = {
    'members': True,
    'special-members': '__init__',
    'undoc-members': True
}

# -- Project information -----------------------------------------------------

project = 'someProject'
copyright = '2020'
author = ''

# The short X.Y version
version = ''
# The full version, including alpha/beta/rc tags
release = '0.0.1'

# -- General configuration ---------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.doctest',
    'sphinx.ext.todo',
    'sphinx.ext.coverage',
    'sphinx.ext.mathjax',
    'sphinx.ext.viewcode',
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'

# The master toctree document.
master_doc = 'index'

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = None

# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'

# Theme options are theme-specific and customize the look and feel of a theme
# further.  For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
#
# The default sidebars (for documents that don't match any pattern) are
# defined by theme itself.  Builtin themes are using these templates by
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
# 'searchbox.html']``.
#
# html_sidebars = {}

# -- Options for HTMLHelp output ---------------------------------------------

# Output file base name for HTML help builder.
htmlhelp_basename = 'MyPackagedoc'

# -- Options for LaTeX output ------------------------------------------------

latex_elements = {
    # The paper size ('letterpaper' or 'a4paper').
    #
    # 'papersize': 'letterpaper',

    # The font size ('10pt', '11pt' or '12pt').
    #
    # 'pointsize': '10pt',

    # Additional stuff for the LaTeX preamble.
    #
    # 'preamble': '',

    # Latex figure (float) alignment
    #
    # 'figure_align': 'htbp',
}

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
#  author, documentclass [howto, manual, or own class]).
latex_documents = [
    (master_doc, 'mypackage.tex', 'mypackage Documentation',
     'Undisclosed_Author', 'manual'),
]

# -- Options for manual page output ------------------------------------------

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
    (master_doc, 'mypackage', 'mypackage Documentation',
     [author], 1)
]

# -- Options for Texinfo output ----------------------------------------------

# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
#  dir menu entry, description, category)
texinfo_documents = [
    (master_doc, 'mypackage', 'mypackage Documentation',
     author, 'mypackage', 'One line description of project.',
     'Miscellaneous'),
]

# -- Options for Epub output -------------------------------------------------

# Bibliographic Dublin Core info.
epub_title = project

# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#
# epub_identifier = ''

# A unique identification for the text.
#
# epub_uid = ''

# A list of files that should not be packed into the epub file.
epub_exclude_files = ['search.html']

# -- Extension configuration -------------------------------------------------

# -- Options for todo extension ----------------------------------------------

# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = True

[gunthergl]

Even if their _init_ is not documented, NeMo does seem to get that running, I take a look how they do it. It is not solved by the following:

class classB(LightningModule, object):
    def __init__(self, a: int):
        """
        Args:
            a:
                asdf
        """
        pass

[gunthergl]

So, NeMo's .rst file is doing that by writing .. autoclass:: explicitely in their documentation source.

Edit: remove autodoc_default_options from conf.py to make the problem more explicit.

# So this automodule FAILS to document classB
.. automodule:: ccc.models.exampleClass
   :members:
   :special-members: __init__
   :undoc-members:
   :show-inheritance:

# If I call classB explicitely, documentation is made
.. autoclass:: ccc.models.exampleClass.classB
   :special-members: __init__
   :show-inheritance:

[gunthergl]

I checked high-starred dependents of pytorch-lightning. Either they use .. autoclass::, or e.g. asteroid has the same issue @mpariente with the asteroid.engine.system - module.

[gunthergl]

So, I have found it is not only a pytorch-lightning issue by trying to document the LightningModule by myself.

from abc import ABC, abstractmethod
from torch.nn import Module

from pytorch_lightning.core.grads import GradInformation
from pytorch_lightning.core.hooks import ModelHooks
from pytorch_lightning.core.saving import ALLOWED_CONFIG_TYPES, PRIMITIVE_TYPES, ModelIO
from pytorch_lightning.utilities.device_dtype_mixin import DeviceDtypeModuleMixin

# original Class:
# class LightningModule(ABC, DeviceDtypeModuleMixin, GradInformation, ModelIO, ModelHooks, Module):

# This gets documented
class ABCmodule(ABC):
    """
    mydocumentation ABCmodule
    """

# The following two classes are not documented.
class PTLmodule(DeviceDtypeModuleMixin):
    """
    mydocumentation PTLmodule
    """

class TORCHmodule(Module):
    """
    mydocumentation TORCHmodule
    """

PyTorch documents Module here under Containers. It uses .. autosummary:: but also some kind of :template:-directive which might solve the problem.

However, I have more urgent, actual code problems. I feel at some point I will make the documentation by hand anyways so I will go with explicitely documenting with .. autoclass:: ccc.models.exampleClass.classB

Would be great though if someone with more knowledge of sphinx could jump in!

[stale[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!

[ydcjeff]

@gunthergl does .. automodule:: work now?

[stale[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!