Open agriyakhetarpal opened 1 month ago
Quite strange. The documentation is building without any problems locally...
Edit: I can reproduce locally if I delete a few of the previously generated files – this is coming from jupyterlite-sphinx
. I think that Read the Docs is using a cached version of the documentation and is not purging the files properly, or it isn't allowing the subprocess to execute at the config-inited
builder hook.
Edit, again: they were failing because the jupytext
command was failing silently and was not converting any notebooks. Removing the hardcoded path fixed it.
I added a basic extension to clean up the generated IPython notebooks in 6bc9c601d73b1116058db3a44e2437c85fc9c02f using nbformat
, works quite well. I am now exploring a method to add the directives based on https://github.com/PyWavelets/pywt/pull/741#discussion_r1619280967 automatically – it requires some configuration but I feel that it can be achieved at the source-read
event before the Markdown files are processed.
Processing Markdown programmatically doesn't bear good fruit – it would have been easier if we used IPyNB, and we don't plan to. I would suggest that we should merge this for now, @rgommers.
The issue with Markdown was that there isn't a clean way to add both the Jupytext frontmatter and the notebook directives and that requires some obtrusive, prone-to-failure file manipulation by processing the contents as a string (we can keep the notebook directives at the bottom and the frontmatter intact, but it's easier to provide the download buttons at the top of the page rather than at the bottom).
Weird, the local docs build and RTD both produce cleaner downloadable notebooks, but the NotebookLite directive on RTD doesn't have the cleaner notebooks (they still have the Sphinx directives). This does not happen locally. I think RTD has some caching troubles, because I removed the redundant module headings from the notebooks (such as (reg-dwt-idwt)=
and so on), but they are still there on RTD.
Edit: some notebooks are as expected, and some don't.
Edit 2: this is most likely the case of a cached build or some updates that don't trickle down the RTD, the logs say that all the notebooks are converted. I can't reproduce locally, and I feel that the issue will go away automatically on a fresh build.
I had a revelation just now and I think 07a09267193c3e0e7ba58ec4d2d1e679a9e65160 should fix the synchronisation issue for good – the RTD build is just buggy for some reason or the other and the local documentation builds where the issue doesn't arise can be trusted upon. Ready for review and further proceedings.
Tagging @melissawm and @steppi for a review as discussed during the 14/06/2024 interactive docs meeting – thanks, both!
This seems to work really well- I love the buttons and dropdowns.
How flexible can we make this to share with other projects? Ideally, we could have an extension (or as a part of jupyterlite-sphinx) that adds the preprocess notebooks event, common settings and looks without having to turn them on manually for each project?
This seems to work really well- I love the buttons and dropdowns.
How flexible can we make this to share with other projects? Ideally, we could have an extension (or as a part of jupyterlite-sphinx) that adds the preprocess notebooks event, common settings and looks without having to turn them on manually for each project?
I had a discussion with @agriyakhetarpal offline where we discussed how this preprocessing could be added to jupyterlite-sphinx. I think he's working on it.
Description
The Usage Examples section, following #728 and as requested in https://github.com/PyWavelets/pywt/pull/737#discussion_r1589709771, is rendered interactive through the changes in this PR through the use of
jupyterlite-sphinx
and Markdown-based notebooks that are executed by MyST-NB. Please read below for a granular overview of all of these details:Which issue does this PR solve/reference?
Addresses a part of #706
Key changes made
.rst
based files underdoc/source/regression
converted to Markdown and reformatted as notebooksconf.py
jupyterlite-sphinx
to run the notebooks under WASM in the same tab (and something like https://github.com/jupyterlite/jupyterlite-sphinx/pull/165 can be added for this directive as well).ipynb
-based notebooks that get generated during the process in order to keep the documentation build warning-free. Generating them at build time is for theNotebookLite
directive to be able to access them and load them, since the NotebookLite directive currently does not load.md
files or notebook files in other formats. i. I imagine this will be helpful for https://github.com/scipy/scipy/pull/20303 as well where it is required to load the notebooks in an interactive manner under a specific folder. The notebooks are not executed by Jupytext at the time of conversion, and therefore, based on past experiments, it takes ~10 seconds to convert 30 or so notebooks.Additional context
This is just a pilot run of how notebook-based examples can be configured for Sphinx-based documentation websites, so there are a few corner cases that I have noticed so far:
conf.py
just like how theTryExamples
directive can be configured.A brief to-do list
Besides the points mentioned above, smaller tasks can be looked into:
.md
and.ipynb
)