Open morcef opened 2 years ago
mansenfranzen
commented
2 years ago Sorry for letting you wait for so long. I have been busy otherwise. And thanks for opening the issue anyway.
The issue that you've been describing has bothered me for some time, too. It would be great to easily identify the actual model that causes autodoc_pydantic to fail. I will put it on the roadmap for the 1.9.0 release.
j-carson
commented
1 year ago I am hitting this bug in _sort_summary_list, except that it is trying to compare int and None.
Would you be open to a temporary workaround pull-request so that the tool doesn't crash?
Documentation with the fields in a different sort order than intended is better than documentation that doesn't build at all.
Two workaround implementations I thought of:
At this point:
tag_order = self.analyzer.tagorder.get(name_with_class) if tag_order is None, return a dummy value (-9999 to put these near the top or 9999 to put them near the end)
At this point
return sorted(names, key=sort_func) Put the sort in a try-block. If the sort throws an exception, just pass the input "names" back instead of a sorted list
Cielquan
commented
1 year ago I am running into the same issue. For me the reason seems to be inheritance.
I have a model RstcheckConfigFile (source) which inherits from pydantic.BaseModel and has some fields and custom validators. Then I have a second Model RstcheckConfig (source) which inherits from RstcheckConfigFile and only adds some more fields.
The aforementioned sort function then gets the name of a validator function from the first model while documentiong the second model and constructs the full name with the name of the second model. This function does not exist in code as it only exists through inheritance. Therefore it cannot be found in self.analyzer.tagorder and the error occurs.
EDIT: Inheritance seems to be another issue altogether I guess, because after I switched from "bysource" to "alphabetical" I get these errors for the code mentioned above:
/home/krys/Projects/.contributions/rstcheck-core/src/rstcheck_core/config.py:docstring of rstcheck_core.config.RstcheckConfig:1: WARNING: py:obj reference target not found: rstcheck_core.config.RstcheckConfig.ignore_messages
/home/krys/Projects/.contributions/rstcheck-core/src/rstcheck_core/config.py:docstring of rstcheck_core.config.RstcheckConfig:1: WARNING: py:obj reference target not found: rstcheck_core.config.RstcheckConfig.ignore_directives
/home/krys/Projects/.contributions/rstcheck-core/src/rstcheck_core/config.py:docstring of rstcheck_core.config.RstcheckConfig:1: WARNING: py:obj reference target not found: rstcheck_core.config.RstcheckConfig.ignore_languages
/home/krys/Projects/.contributions/rstcheck-core/src/rstcheck_core/config.py:docstring of rstcheck_core.config.RstcheckConfig:1: WARNING: py:obj reference target not found: rstcheck_core.config.RstcheckConfig.ignore_roles
/home/krys/Projects/.contributions/rstcheck-core/src/rstcheck_core/config.py:docstring of rstcheck_core.config.RstcheckConfig:1: WARNING: py:obj reference target not found: rstcheck_core.config.RstcheckConfig.ignore_substitutions
/home/krys/Projects/.contributions/rstcheck-core/src/rstcheck_core/config.py:docstring of rstcheck_core.config.RstcheckConfig:1: WARNING: py:obj reference target not found: rstcheck_core.config.RstcheckConfig.report_levelI only have these flags set:
autodoc_pydantic_model_erdantic_figure = False
autodoc_pydantic_model_show_json = FalseEDIT2: I am using the lazy way with automodule.
mansenfranzen
commented
10 months ago Thanks @morcef @Cielquan for providing more detail! @j-carson started a PR via #190. Currently, we are lacking a minimal reproducible example that covers the incorrect behaviour/s. This will be required in the test suite. Otherwise, a reliable fix is going to be difficult. Any additional input is highly appreciated!
j-carson
commented
10 months ago @morcef @Cielquan The PR https://github.com/mansenfranzen/autodoc_pydantic/pull/190 is getting into better shape now -- if you can test it with your docs that are breaking let me know!
Cielquan
commented
8 months ago main in rstcheck-corepip install git+https://github.com/j-carson/autodoc_pydantic.git@issue_137conf.py file
extensions.append("sphinxcontrib.autodoc_pydantic")
autodoc_pydantic_model_erdantic_figure = False
autodoc_pydantic_model_show_json = Falsetox -e docs and got this output:
$ tox -e docs
.pkg: _optional_hooks> python /home/user/.local/pipx/venvs/tox/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python /home/user/.local/pipx/venvs/tox/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_sdist> python /home/user/.local/pipx/venvs/tox/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta
docs: install_package> python -I -m pip install --force-reinstall --no-deps /home/user/Projects/.contributions/rstcheck-core/.tox/.tmp/package/355/rstcheck-core-1.2.1.dev35+g9d7a9b8c73b0.d20240303.tar.gz
docs: commands[0]> sphinx-build --color -b html -aE docs/source docs/build/html
Running Sphinx v7.2.6
/home/user/Projects/.contributions/rstcheck-core/.tox/docs/lib/python3.11/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py:21: RemovedInSphinx80Warning: The alias 'sphinx.util.typing.stringify' is deprecated, use 'sphinx.util.typing.stringify_annotation' instead. Check CHANGES for Sphinx API modifications.
from sphinx.util.typing import get_type_hints, stringify
Initializing Spelling Checker 8.0.0
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from objects.pattern.inv...
loading intersphinx inventory from objects.pydantic.inv...
loading intersphinx inventory from objects.sphinx.inv...
Creating file /home/user/Projects/.contributions/rstcheck-core/docs/source/autoapidoc/rstcheck_core.rst.
building [mo]: all of 0 po files
writing output...
building [html]: all source files
updating environment: [new config] 16 added, 0 changed, 0 removed
/home/user/Projects/.contributions/rstcheck-core/.tox/docs/lib/python3.11/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py:137: RemovedInSphinx80Warning: The tuple interface of ObjectMember is deprecated. Use (obj.__name__, obj.object) instead.
return {x[0] for x in self._documenter.get_object_members(True)[1]}
reading sources... [100%] workflows/releases
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
copying assets... copying static files... done
copying extra files... done
done
writing output... [100%] workflows/releases
generating indices... genindex py-modindex done
highlighting module code... [100%] rstcheck_core.types
writing additional pages... search done
dumping search index in English (code: en)... done
dumping object inventory... done====================== slowest reading durations ======================= 0.572 autoapidoc/rstcheck_core 0.043 installation 0.038 _badges 0.022 changelog 0.021 usage/config build succeeded.
The HTML pages are in docs/build/html. docs: commands[1]> python -c 'from pathlib import Path; index_file = Path(r"/home/user/Projects/.contributions/rstcheck-core")/"docs/build/html/index.html"; print(f"DOCUMENTATION AVAILABLE UNDER: {index_file.as_uri()}")' DOCUMENTATION AVAILABLE UNDER: file:///home/user/Projects/.contributions/rstcheck-core/docs/build/html/index.html .pkg: _exit> python /home/user/.local/pipx/venvs/tox/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta docs: OK (4.07=setup[2.24]+cmd[1.81,0.02] seconds) congratulations :) (4.11 seconds)
So it build the docs.
But there are two deprecation warnings pointing to autodoc_pydantic:/home/user/Projects/.contributions/rstcheck-core/.tox/docs/lib/python3.11/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py:21: RemovedInSphinx80Warning: The alias 'sphinx.util.typing.stringify' is deprecated, use 'sphinx.util.typing.stringify_annotation' instead. Check CHANGES for Sphinx API modifications. from sphinx.util.typing import get_type_hints, stringify
/home/user/Projects/.contributions/rstcheck-core/.tox/docs/lib/python3.11/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py:137: RemovedInSphinx80Warning: The tuple interface of ObjectMember is deprecated. Use (obj.__name__, obj.object) instead. return {x[0] for x in self._documenter.get_object_members(True)[1]}
PS: I added a branch with the above changes: https://github.com/rstcheck/rstcheck-core/tree/autodoc-pydantic-testing
EDIT: update git link
EDIT2: fix typoHey @Cielquan these warnings look like your tox environment is not picking up the correct autodoc_pydantic as I did fix first warning and the second one is merged but just not on the pypi release yet:
stringify fix is here https://github.com/j-carson/autodoc_pydantic/blob/f55e3207da9d228e0a3675ff2f6c91b214b0b7dd/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py#L23
using .__name__ is here: https://github.com/j-carson/autodoc_pydantic/blob/f55e3207da9d228e0a3675ff2f6c91b214b0b7dd/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py#L143
Remember that you can't just put autodoc_pydantic in your local virtualenv and call it good because tox creates it's own virtualenvs (in .tox/docs in this case, but it will be in .tox/whatever_tox_env_name_is) and can be picking up the wrong version of autodoc_pydantic if you are not careful.
Cielquan
commented
8 months ago Oh yeah. My bad. Did not think this through. Will test again in the coming days when I got time.
mansenfranzen
commented
8 months ago @all-contributors please add @Cielquan for bug
allcontributors[bot]
commented
8 months ago @mansenfranzen
I've put up a pull request to add @Cielquan! :tada:
mansenfranzen
commented
8 months ago @all-contributors please add @morcef for bug
allcontributors[bot]
commented
8 months ago @mansenfranzen
I've put up a pull request to add @morcef! :tada:
mansenfranzen
commented
8 months ago Before merging the related PR, it would be great if you could test the changed behavior. To do so, please install the current dev release in your doc-building-environment via pip install git+https://github.com/mansenfranzen/autodoc_pydantic.git@v2.1.0-a.1 and rebuild your docs.
Please feel free to reopen if any problem remains. I will release the new version upcoming Monday.
Cielquan
commented
7 months ago I can confirm the issue is gone.
juhlinm
commented
7 months ago I'm still seeing the issue. Using autodoc_pydantic 2.1.0. I'm not sure if there might be some other package that is causing the issue.
Running Sphinx v7.2.6
making output directory... done
myst v2.0.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=set(),
disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'),
ref_domains=None, fence_as_directive=set(), number_code_blocks=[], title_to_header=False,
heading_anchors=0, heading_slug_func=None, html_meta={}, footnote_transition=True,
words_per_minute=200, substitutions={}, linkify_fuzzy_links=True, dmath_allow_labels=True,
dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True,
mathjax_classes='tex2jax_process|mathjax_process|math|output_area', enable_checkboxes=False,
suppress_warnings=[], highlight_code_blocks=True)
building [mo]: targets for 0 po files that are out of date
writing output...
building [html]: targets for 5 source files that are out of date
updating environment: [new config] 5 added, 0 changed, 0 removed
Exception occurred:
File "/usr/local/lib/python3.10/site-packages/sphinxcontrib/autodoc_pydantic/directives/
autodocumenters.py", line 594, in _sort_summary_list
return sorted(names, key=sort_func)
TypeError: '<' not supported between instances of 'NoneType' and 'NoneType'Do you have a small test case?
@juhlinm When you have a test case, maybe open a new ticket since this one is closed
mansenfranzen
commented
7 months ago @juhlinm I added an improved error message to the corresponding function in #244 which should provide the name of the model causing the error.
It would be great if you could run your example with the modified version. To do so, please install the current dev release in your doc-building-environment via pip install git+https://github.com/mansenfranzen/autodoc_pydantic.git@v2.1.1rc3 and rebuild your docs.
juhlinm
commented
7 months ago @mansenfranzen We got an error message, which might be able to help us get further. Thanks.
Exception occurred:
File "/usr/local/lib/python3.10/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py", line 571, in sort_func
raise ValueError(msg)ValueError: Field item in DynamicDependency not found in tagorderEDIT: Here is a minimal example of our code, that will lead to the issue.
from abc import ABC
from typing import Any, ClassVar, Dict, Generic, Optional, Type, TypeVar
from pydantic import BaseModel, ConfigDict
NO_TAG = "__no_tag__"
class BaseItem(BaseModel, ABC):
__tag__: str = ""
__group__: str = ""
__subtypes__: Dict[str, Type['BaseItem']] = dict()
model_config = ConfigDict(extra="forbid", arbitrary_types_allowed=True)
def __init_subclass__(cls: Type['BaseItem'], *args: Any, **kwargs: Any) -> None:
if "tag" in kwargs:
tag = kwargs["tag"]
if tag == NO_TAG:
return
cls.__tag__ = tag if tag else cls.__name__.lower()
cls.__subtypes__[cls.__tag__] = cls
else:
if BaseItem in cls.__bases__:
cls.__subtypes__ = dict()
ItemT = TypeVar("ItemT", bound=BaseItem)
class DynamicBaseItem(BaseModel, Generic[ItemT]):
__over__: ClassVar[Any]
item: ItemT
"""The BaseItem that is wrapped."""
model_config = ConfigDict(extra="forbid")
def __init_subclass__(cls: Type['DynamicBaseItem[ItemT]'], over: Type[ItemT] = None) -> None:
if over:
cls.__over__ = over # type: ignore
class Dependency(BaseItem):
async def run(self) -> Any:
pass
class DynamicDependency(DynamicBaseItem[Dependency], over=Dependency):
name: Optional[str] = None
'ValueError: Field item in DynamicDependency not found in tagorder' 'item' seems to be defined in the parent class, but there is some meta-programming going on here, which I'm not fully familiar with. Maybe that affects something.
j-carson
commented
7 months ago @juhlinm I'm having problems with causing your test example to fail:
Could I get your sphinx/autodoc configuration settings please? Which of the members are you trying to document? Are you turning on inherited/special/private members? Which versions of pydantic are you on?
juhlinm
commented
7 months ago @j-carson Sure, here is the conf.py used:
from typing import List
project = "XXX"
copyright = "XXX"
author = "XXX"
release = "0.1"
extensions: List[str] = [
"sphinx.ext.autodoc", "sphinx.ext.napoleon", "sphinx.ext.viewcode", "sphinx.ext.graphviz", "myst_parser",
'sphinxcontrib.autodoc_pydantic'
]
source_suffix = {
'.rst': 'restructuredtext',
'.md': 'markdown',
}
html_static_path = ['static']
exclude_patterns = [
"install.py",
"conf.py",
"**/out",
"**/.coverage",
"**/.pytest_cache",
".pytest_cache",
"**/.vscode",
"**/*.pyc",
"**/__pycache__",
"**/*.egg-info",
"**/.eggs",
"**/dist",
"**/build",
"**/.virtualenv",
".virtualenv",
"**/lib",
"**/.idea",
"**/.cache",
"**/coverage",
"**/.mypy_cache",
"**/version.txt",
"**/pytest_coverage",
"**/pytest_*",
"**/.venv",
"**/.vagrant",
"**/log.html",
"**/output.xml",
"**/report.html",
]
html_theme = "sphinx_rtd_theme"
add_module_names = False
python_use_unqualified_type_names = True
autodoc_pydantic_model_undoc_members = False
autodoc_pydantic_model_show_json = False
autodoc_pydantic_field_list_validators = False
autodoc_pydantic_model_show_validator_summary = False
autodoc_pydantic_field_show_constraints = True
autodoc_pydantic_field_doc_policy = "both"
autodoc_pydantic_model_summary_list_order = "bysource"
def setup(app) -> None: # type: ignore
app.add_css_file('style.css')The error occurs when trying to document the field item of DynamicDependency (inherited).
We have inherited members turned on for BaseModel. Do we need it for DynamicBaseItem as well?
.. autopydantic_model:: proj.DynamicDependency
:inherited-members: BaseModelPydantic 2.6.4
afdaniele
commented
6 months ago Hi,
I'm seeing the same error. It looks like it only appears when the directive configuration :model-summary-list-order: bysource is used together with :inherited-members: pydantic.BaseModel. If I leave model-summary-list-order to its default configuration it builds without errors.
Hope this helps.
mansenfranzen
commented
6 months ago @afdaniele Thanks for sharing your insights!
Do you have a reproducible example, too? This would be highly appreciated to improve our test coverage for this bug.
afdaniele
commented
6 months ago Unfortunately, I came across this issue only as part of a big project developed using jupyter-book, a wrapper around sphinx, so, a long way from a simple sphinx example to share.
If it can be of any help though, the model I'm trying to document is relatively simple and has a hierarchy of 3 levels.
It is a library of messages that are serialized and exchanged over the network. There is a base class called BaseMessage that inherits from pydantic.BaseModel. A class Sensor(BaseMessage) and then finally a class Camera(Sensor).
While BaseMessage does not have fields, both Sensor and Camera do, and so if I do,
.. autopydantic_model:: duckietown_messages.sensors.camera.Camera
:model-summary-list-order: bysource
:inherited-members: pydantic.BaseModelI get the error above. While,
.. autopydantic_model:: duckietown_messages.sensors.camera.Camera
:inherited-members: pydantic.BaseModelworks fine, except for the unwanted alphabetical order of the members. Every other configuration parameter is left at the default value. Hope this helps.
juhlinm
commented
6 months ago Yeah, removing the :inherited-members: pydantic.BaseModel from the .rst-file made our documentation finish generation.
j-carson
commented
5 months ago @mansenfranzen Were you ever able to turn any of the examples into a failing test?
Hi,
I have encountered a simmilar issue as discussed in #78. My Pydantic models have fields with type of other Pydantic models, and when using
autodoc_pydantic_model_summary_list_order = 'bysource'I still seem to get the same error.Is there any other way to find the exact model that fails, except for the elimination method?
I am not sure what more should be provided to find my edge case, but here is the Sphinx traceback: