search

mansenfranzen / autodoc_pydantic

Seamlessly integrate pydantic models in your Sphinx documentation.
MIT License
158 stars 27 forks source link

KeyError: 'created_by' when building documentation #128

Closed PipeKnight closed 2 years ago

PipeKnight commented 2 years ago

Full error trace:

# Sphinx version: 5.0.0
# Python version: 3.8.13 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 3.1.2
# Last messages:
#   reading sources... [ 50%] reference/app.crud.crud_role
#   reading sources... [ 51%] reference/app.crud.crud_team
#   reading sources... [ 53%] reference/app.crud.crud_user
#   reading sources... [ 54%] reference/app.db
#   reading sources... [ 56%] reference/app.db.init_db
#   reading sources... [ 57%] reference/app.db.session
#   reading sources... [ 59%] reference/app.initial_data
#   reading sources... [ 60%] reference/app.main
#   reading sources... [ 62%] reference/app.models
#   reading sources... [ 63%] reference/app.models.group
# Loaded extensions:
#   sphinx.ext.mathjax (5.0.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/alabaster/__init__.py
#   sphinx.ext.todo (5.0.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/todo.py
#   sphinx.ext.viewcode (5.0.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/viewcode.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (5.0.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (5.0.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.napoleon (5.0.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/napoleon/__init__.py
#   sphinxcontrib.apidoc (0.3.0) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/apidoc/__init__.py
#   sphinxcontrib.autodoc_pydantic (1.7.1) from /Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/__init__.py
Traceback (most recent call last):
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app.build(args.force_all, filenames)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/application.py", line 329, in build
    self.builder.build_update()
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 288, in build_update
    self.build(to_build,
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 302, in build
    updated_docnames = set(self.read())
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 409, in read
    self._read_serial(docnames)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 430, in _read_serial
    self.read_doc(docname)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 483, in read_doc
    publisher.publish()
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/io.py", line 103, in read
    self.parse()
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/parsers.py", line 78, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
    return method(self, expmatch)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
    return self.run_directive(
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
    result = directive_instance.run()
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/directive.py", line 148, in run
    documenter.generate(more_content=self.content)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 955, in generate
    self.document_members(all_members)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 831, in document_members
    documenter.generate(
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 1788, in generate
    return super().generate(more_content=more_content,
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 955, in generate
    self.document_members(all_members)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py", line 164, in document_members
    super().document_members(*args, **kwargs)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 1779, in document_members
    super().document_members(all_members)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 831, in document_members
    documenter.generate(
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 945, in generate
    self.add_directive_header(sig)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/directives/options/composites.py", line 259, in wrapped
    result = func(*args, **kwargs)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py", line 539, in add_directive_header
    self.add_default_value_or_marker()
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py", line 586, in add_default_value_or_marker
    if self.needs_required_marker:
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py", line 549, in needs_required_marker
    is_required = self.pydantic.inspect.fields.is_required(field_name)
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/inspection.py", line 188, in is_required
    return self.get(field_name).required
  File "/Users/pipeknight/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sphinxcontrib/autodoc_pydantic/inspection.py", line 137, in get
    return self.attribute[name]
KeyError: 'created_by'

My conf file:

# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
import os
import sys

# Also we may load a dotenv file here
sys.path.insert(0, os.path.abspath('.'))

# -- Project information -----------------------------------------------------

project = 'fastapi-alembic-sqlmodel-async'
copyright = '2022, -'
author = '-'

# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    'sphinx.ext.todo',
    'sphinx.ext.viewcode',
    'sphinx.ext.autodoc',
    'sphinx.ext.napoleon',
    'sphinxcontrib.apidoc',
    'sphinxcontrib.autodoc_pydantic',
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# Apidoc options
apidoc_module_dir = '../fastapi-alembic-sqlmodel-async'
apidoc_output_dir = 'reference'
apidoc_excluded_paths = ['tests', 'kafka_test']
apidoc_separate_modules = True
# apidoc_extra_args = ['-d 2']

# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
html_theme = 'alabaster'
# html_theme = 'sphinx-rtd-theme'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

I have several models in my app with field named created_by, for example:

from sqlmodel import Field, Relationship, SQLModel

class Group(SQLModel, table=True):
    id: Optional[int] = Field(default=None, nullable=False, primary_key=True)
    updated_at: Optional[datetime]
    created_at: Optional[datetime]
    created_by_id: Optional[UUID] = Field(default=None, foreign_key="user.id")
    created_by: "User" = Relationship(  # noqa: F821
        sa_relationship_kwargs={
            "lazy": "select",
            "primaryjoin": "Group.created_by_id==User.id",
        }
    )

And for some reason this broke my docs generation. If I comment 'sphinxcontrib.autodoc_pydantic' in conf.py extensions, than everything works

PipeKnight commented 2 years ago

So maybe it's the problem with supporting sqlmodel library in your extension, which is something like a bridge between pydantic and sqlalchemy

mansenfranzen commented 2 years ago

Thanks for reporting the issue here! It is very likely related to sqlmodel. There is a similar issue here #124. I will take a look at it soonish.

mansenfranzen commented 2 years ago

Okay, this is related to what a Relationship represents in sqlmodel. In fact, it is not a field (or column) of a pydantic model (or sqlalchemy table).

autodoc_pydantic currently assumes all class attributes of a parent pydantic model to be a pydantic field which is simply wrong in this case. It will require an additional, more explicit check if a class attribute of a pydantic model parent is truly a pydantic field. Luckily, this is not too difficult to implement and it should actually prevent further unwanted side effects.

mansenfranzen commented 2 years ago

This bug is addressed via #130 .

Before merging the related PR, it would be great if you could test the bugfix on your site. To do so, please install the current dev release in your doc-building-environment via pip install git+https://github.com/mansenfranzen/autodoc_pydantic.git@v1.7.2-a.1 and rebuild your docs.

mansenfranzen commented 2 years ago

@all-contributors please add @PipeKnight for bug

allcontributors[bot] commented 2 years ago

@mansenfranzen

I've put up a pull request to add @PipeKnight! :tada:

mansenfranzen commented 2 years ago

@PipeKnight This should be fixed with the current v1.7.2 release of today. Feel free to reopen if the issue persists.