apache / iceberg-python

Apache PyIceberg
https://py.iceberg.apache.org/
Apache License 2.0
590 stars 218 forks source link

Typedefs such as Identifier, Properties, and RecursiveDict are not hyperlinked in the generated documentation #1529

Closed rodrigc closed 2 weeks ago

rodrigc commented 3 weeks ago

Apache Iceberg version

0.8.1 (latest release)

Please describe the bug 🐞

I was working on writing some code with pyiceberg, and I noticed that Identifier is not hyperlinked in the docs.

For example here: https://py.iceberg.apache.org/reference/pyiceberg/catalog/#pyiceberg.catalog.Catalog.create_table_if_not_exists

It would be useful to get types that are defined in the Pyiceberg library to be hyperlinked in the documentation. This would also apply for types such as [Properties (https://github.com/apache/iceberg-python/blob/e8e2c91379b4b4eb501b7b60355837af73c7c35e/pyiceberg/typedef.py#L77) and RecursiveDict.

Willingness to contribute

kevinjqliu commented 3 weeks ago

Thanks for reporting this! Its helpful to have those variables available and linked in the docs.

For context, we use mkdocstrings to generate docs from the source code. I think the current problem is that variables such as Identifier , Properties and RecursiveDict are not recognized in the docs under typedef. And because its not recognized, the rest of the docs are not able to hyperlink to it.

A potential solution is to add docstring to the variables. The docstrings should force mkdocstrings to recognize these as part of the docs.

from https://mkdocstrings.github.io/python/#features:

Support for documented attributes: attributes (variables) followed by a docstring (triple-quoted string) will be recognized by Griffe in modules, classes and even in init methods.

kevinjqliu commented 3 weeks ago

BTW you can run a local version of the docs website by running make docs-serve. Hope this helps! And happy to answer any follow up questions

rodrigc commented 3 weeks ago

The docs are generated via the scripts here: https://github.com/apache/iceberg-python/tree/main/mkdocs

I did the following:

  1. Put debugging in https://github.com/apache/iceberg-python/blob/main/mkdocs/gen_doc_stubs.py to see what files are being parsed
  2. Ran:
    make docs-install
    make docs-serve
rodrigc commented 3 weeks ago

Your advice led me down the correct path to identify the problem.

mkdocstrings relies on the presence of docstrings to recognize and document variables, types, and other attributes. Since those types did not have docstrings, they were not picked up by mkdocstrings.

I submitted this PR: https://github.com/apache/iceberg-python/pull/1530 and when I ran make docs-serve locally, I could now see that Identifier was hyperlinked in the docs for create_table_if_not_exists().