Use sphinx-autoapi to generate API documentation

valeriupredoi commented 4 weeks ago

Description

Closes #3644
Link to documentation:
- *

Before you get started

[ ] ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[ ] 🛠 This pull request has a descriptive title
[ ] 🛠 Code is written according to the code quality guidelines
[ ] 🛠 Documentation is available
[ ] 🛠 Tests run successfully
[ ] 🛠 The list of authors is up to date
[ ] 🛠 Any changed dependencies have been added or removed correctly
[ ] 🛠 All checks below this pull request were successful

New or updated recipe/diagnostic

[ ] 🧪 Recipe runs successfully
[ ] 🧪 Recipe is well documented
[ ] 🧪 Figure(s) and data look as expected from literature
[ ] 🛠 Provenance information has been added

New or updated data reformatting script

[ ] 🛠 Documentation is available
[ ] 🛠 The dataset has been added to the CMOR check recipe
[ ] 🛠 The dataset has been added to the shared data pools of DKRZ and Jasmin by the @ESMValGroup/OBS-maintainers team
[ ] 🧪 Numbers and units of the data look physically meaningful

To help with the number of pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

valeriupredoi commented 3 weeks ago

@bouweandela have you actually run the API docs builds with sphinx-autoapi? I have just now, locally, and it's incredibly slow and spits out a gazillion formatting errors a la:

/home/valeriu/ESMValTool/doc/sphinx/source/autoapi/single_model_diagnostics/index.rst:179: ERROR: Unexpected indentation.
/home/valeriu/ESMValTool/doc/sphinx/source/autoapi/utils/index.rst:130: ERROR: Unexpected indentation.
/home/valeriu/ESMValTool/doc/sphinx/source/autoapi/utils/index.rst:132: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/valeriu/ESMValTool/doc/sphinx/source/autoapi/ww09_esmvaltool/index.rst:41: ERROR: Unexpected indentation.

valeriupredoi commented 3 weeks ago

also any clue how to tell ReadTheDocs we don't want a local build of the package? It complains it's missing the pkg metadata since the darn thing was not pip installed, but we don't want that, that's exactly what we're trying to avoid with autoapi https://readthedocs.org/projects/esmvaltool/builds/24658734/

bouweandela commented 3 weeks ago

We probably need a different way of getting the version than importing the whole package (which fails because it is not installed):

https://github.com/ESMValGroup/ESMValTool/blob/ae00a53aac62cbbf2b21c425b92ae7eed7cafffb/doc/sphinx/source/conf.py#L26

bouweandela commented 3 weeks ago

have you actually run the API docs builds with sphinx-autoapi?

No

I have just now

:beers:

it's incredibly slow

How slow is incredibly slow? The current build on readthedocs takes 15 minutes.

it spits out a gazillion formatting errors

Are those for modules that are part of the public API?

valeriupredoi commented 3 weeks ago

let's see how slow it is on RtD - when it runs, it took me some 10min and I killed it; indeed, all manners of modules that should have private API, think that needs configuring not to go through those, need to see their configuration settings. Ah cheers for the __version__ pointer, have completely overlooked that! :beer:

valeriupredoi commented 3 weeks ago

well good news! It ran and it ran in 6min :partying_face: The warnings are annoying though - it's looking for esmvaltool modules in diags - we need to configure it so it doesn't do that; incidentally, RtD's new dashboard is looking nice but it adds a million empty lines to an already very stuffy output https://app.readthedocs.org/projects/esmvaltool/builds/24659446/

valeriupredoi commented 3 weeks ago

OK 396 warnings from intial 470 - I am now excluding diag_scripts - otherwise we get all these sort of warnings:

Argument:
--------
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/ens_eof_kmeans/index.rst:30: ERROR: Unexpected indentation.
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/ens_eof_kmeans/index.rst:31: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/esmvaltool/diag_scripts/autoassess/_plot_mo_metrics/index.rst:187: ERROR: Unexpected indentation.
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/esmvaltool/diag_scripts/autoassess/_plot_mo_metrics/index.rst:188: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/esmvaltool/diag_scripts/autoassess/autoassess_area_base/index.rst:151: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/esmvaltool/diag_scripts/autoassess/autoassess_area_base/index.rst:151: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/esmvaltool/diag_scripts/autoassess/loaddata/index.rst:167: ERROR: Unexpected indentation.
/home/docs/checkouts/readthedocs.org/user_builds/esmvaltool/checkouts/3646/doc/sphinx/source/autoapi/esmvaltool/diag_scripts/emergent_constraints/lif1f2/index.rst:3: CRITICAL: Missing matching underline for section title overline.

valeriupredoi commented 3 weeks ago

OK so this is bugging me:

WARNING: Cannot resolve cyclic import: esmvaltool.diag_scripts.psyplot_diag, esmvaltool.diag_scripts.shared, esmvaltool.diag_scripts.shared._validation, esmvaltool.diag_scripts.shared

bouweandela commented 2 weeks ago

This is not quite ready for review yet: the text from the removed .rst files still needs to be moved to the docstrings of the relevant python modules.

valeriupredoi commented 2 weeks ago

This is not quite ready for review yet: the text from the removed .rst files still needs to be moved to the docstrings of the relevant python modules.

well, you are the only reviewer, and major contributor, so this is ready only when you say so, bud - plop a changes needed blocker and we is good :beer:

valeriupredoi commented 1 week ago

@bouweandela am gonna fix the myriad of conflicts now, then maybe you have a second or two give it a bit more polish, and merge? It don't have to be perfect :grin:

ESMValGroup / ESMValTool