Open JoeZiminski opened 1 month ago
I plan to read and comment soon, but I have a presentation I'm working on at the moment. Feel free to ping me on this @JoeZiminski next week :)
I don't think we should worry about replacing real data with synthetic data. I think it's only 3 tutorials we'd need to make alterations to (motion, neuropixles and quickstart) and there are ways we can make up for the real data (link to seminars/papers which include real data? Make and link to a blog?). Replacing the mearc downloads in the other tutorials with generated data is also simple.
Your solution in #2879 is great, though the build tags solution does sound simpler. Seems like Sphinx suggests something like that here https://sphinx-gallery.github.io/stable/configuration.html#parsing-and-executing-examples-via-matching-patterns ?
Further to the previous meeting, I thought we could discuss here what to do about stucture and automatic building for the documentation.
I think we agreed that:
index.rst
pages that can link to both sphinx-built and non-sphinx-built documentation for How To's and Tutorials (i.e. we will not use the auto-genereated sphinx gallery)In terms of automation options, two possibilities are:
.py
files. At the moment there are two modes -.py
files prefixed withplot_
are run in full when converted to.rst
. Otherwise, the python code is not actually run and the.py
file is converted to.rst
(so you get no plots or code output). It would be cool to use the tag like
sphinx-build -b html doc doc/build -t long-time-buildsthat changes the keywords used to fully build the long-docs. For example it could extend the
plot_prefix to also include
long_build. Then, when the tag is not used, these long-build docs will just be converted to
.rst` only (i.e. without running the python code) and it will be fast. But when properly building the docs, we can use the tag to ensure that they are build with running the python code and images etc. are generated.Handling docs with heavy data / environment requirements
Blockers
.py
file for sphinx-gallery format. That's okay, but where the documentation requires a complex environment / other data it is not possible to the CI to build build so it's also necessary to rewrite parts of these documentation to use synthetic data at the same time as introducing the new build machinery. But, in some cases (drift docs) this changes the interpretation of the data and so a lot of the docs article changes. In this case, the PR like #2879 becomes bloated with sphinx-machinery changes and article-content changes. But I'm not sure of the best way around this.