Closed nathanshammah closed 3 months ago
I'd like to work on this issue. The myST docs themselves show these options on the top right part of the page:
Note that, nicely, it changes the output format to just md and pdf if there is no code,
It seems to me that this part of the conf.py file may configure it
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_title = ""
html_theme = "sphinx_book_theme"
html_logo = "_static/logo-wide.svg"
html_favicon = "_static/logo-square.svg"
html_theme_options = {
"github_url": "https://github.com/executablebooks/myst-nb",
"repository_url": "https://github.com/executablebooks/myst-nb",
"repository_branch": "master",
"home_page_in_toc": True,
"path_to_docs": "docs",
"show_navbar_depth": 1,
"use_edit_page_button": True,
"use_repository_button": True,
"use_download_button": True,
"launch_buttons": {
"binderhub_url": "https://mybinder.org",
"notebook_interface": "classic",
},
}
Bummer... it seems that this configuration is not supported by the pydata theme. I found a hack around this by surfing projects using the theme, in the NetworkX docs, where a button on the top of examples allows one to export the .pynb file.
Hi, I am interested on working this issue but I need to understand this point more clearly: "So, ideally, somehow the Jupyter notebooks should be created somehow (at documentation build or with another script), but still appear in the version of the docs that is hosted on readthedocs. It's a tricky issue :).". Can you explain this point?
Hi @devilkiller-ag, we currently use .md
files for the source code of the documentation. They are better than .ipynb
files for the source code, and so we would like to keep using .md
files in the source code.
So, the optimal solution would be to convert the .md
files to .ipynb
notebooks only during the building process of the documentation.
E.g. by acting in some way in the following configuration files:
https://github.com/unitaryfund/mitiq/blob/master/.readthedocs.yml
https://github.com/unitaryfund/mitiq/tree/master/.github/workflows
https://github.com/unitaryfund/mitiq/blob/master/docs/source/conf.py
https://github.com/unitaryfund/mitiq/blob/master/docs/make.bat
We don't have a clear solution in mind. In fact, the most difficult part of this issue is exploring and proposing a solution, if it exists :-)
Hi, Can we use Jupytext to convert our .md to .ipynb? We can do something like this:
ipynb_file = os.path.splitext(md_file)[0] + '.ipynb'
jupytext.writef(jupytext.read(md_file), ipynb_file, fmt='notebook')
We can create a script that utilizes jupytext to convert the .md files to .ipynb notebooks. And call it in sphinx conf.py file before build.
I don't want to admit how much time I spent on this without non-hacky success, but here is what I learned. I hope it can be useful to others.
As Nate mentioned, the download button is not an option in the pydata theme that Mitiq currently uses. However, the button is a part of the sphinx_book_theme
. The theme is built on top of the pydata theme, and it can by used by implementing the following two steps together:
pydata-sphinx-theme==0.11.0
with sphinx-book-theme==1.0.1
html_theme = "pydata_sphinx_theme"
to html_theme = "sphinx_book_theme"
After making those changes, I recommend the following html_theme_options
on line 314:
html_theme_options = {
"announcement": '<a href="https://unitaryhack.dev/">unitaryHACK</a> is \
coming <b>May 26-Jun 13</b>! Get rewarded for contributing to open source \
quantum software!',
"repository_url": "https://github.com/unitaryfund/mitiq",
"repository_branch": "master",
"use_source_button": False,
"use_repository_button": True,
"use_issues_button": True,
"use_download_button": True,
"use_fullscreen_button": False,
"path_to_docs": "docs/source",
}
Those options do a few things, including adding the download button. Unfortunately, the download only includes the .md
file and not the .ipynb
file. If you start out with a function to convert the .md
files to .ipynb
files, then you get errors in Sphinx, which make it difficult to clean up those generated .ipynb
files automatically, since make
will not run a cleanup script after an exit code 1, and you are left with a bunch of files to cleanup by hand. This is also where things started to feel very hacked together.
Using the sphinx_book_theme
, MyST-NB includes a download button that has both .ipynb
and .md
files, but I cannot for the life of me figure out how they did it after reading through their conf.py
file. Jupyter Book also falls into the same boat, though their configuration is even harder to understand since Sphinx is only part of their setup and they actually use Jupyter Book to build their site. I considered reaching out to their maintainers to ask how they do it, but it's a long weekend in a few countries, and I don't want to still be doing this next week. If someone else gets in touch with them, more power to you.
If you convert notebooks from .md
to .ipynb
, keep in mind that links between books still reference the .md
version. Therefore, the conversion makes the links point to non-existent files, which raises more Sphinx errors.
It's hard to know how changes are being implemented without rerunning the books from a clean state, but that takes forever. Therefore, the best luck I've found for a fast pipeline is to delete all but one or two books while iterating, and then rollback the deletions once you think you have a solution worth testing full-on. This makes make clean html
not take very long at all.
My next two approaches that I am considering trying are:
Makefile
and make.bat
need to mirror each other, so a Makefile
solution would be more work to maintain.I don't really care about a bounty, and now I just want this solved. I hope someone else benefits from what I learned and gets this done in an elegant way.
Happy hacking
Thanks @andre-a-alves, I am going to give this a try.
@devilkiller-ag Take a look at my branch
It copies the source directory to _source, and then it converts .md
files into .ipynb
files, but only if they are Jupyter Notebook files. It also only converts the files if the .md
files have been modified. These decisions were based on:
I still see the following issues that need to be dealt with:
.ipynb
: WARNING: multiple files found for the document
vqe-pyquil-demo.ipynb:230002: WARNING: 'myst' reference target not found: ../guide/zne-3-options.md#extrapolation-methods-factory-objects
b. glossary.ipynb:10018: WARNING: 'myst' reference target not found: ./zne-3-options.md#unitary-folding
c. glossary.ipynb:10018: WARNING: 'myst' reference target not found: ./zne-3-options.md#folding-gates-by-fidelity
d. rem-3-options.ipynb:10005: WARNING: 'myst' reference target not found: rem-5-theory.md#what-is-a-confusion-matrix
e. zne-4-low-level.ipynb:220002: WARNING: 'myst' reference target not found: zne-3-options.md#extrapolation-methods-factory-objects
What I don't like is that it does not give the option to download either .md or .ipynb, and it also can't launch MyBinder, because MyBinder has the .md files and the docs have the .ipynb files.
I am sure there has to be a way to get both files into the download button like MyST-NB does, but I am at a loss as to what that is.
All that being said, I'm going to enjoy the next couple of days of a three-day weekend. Feel free to take my branch and work on the remaining issues if it helps
Thanks @andre-a-alves I will look into your branch and understand the issue.
Excellent writeup @andre-a-alves and hopefully this is helpful for a solution.
Hi @nathanshammah I would also like to work on this issue. @devilkiller-ag can we team up ?
@aryanguptaaa Yeah, that would be great 👍🏻
@nathanshammah Can we team up?
Yes, you can! There was a similar question on Discord and so I copy and paste the answer:
If you do work on a team, ensure you make it clear when opening a PR. You can do this by either ensuring all authors have commits on the branch, or adding a description to the PR with all the authors github handles.
Moreover, of course, you would get a single bounty which must be shared with the team.
Thanks you @andreamari , I will add @aryanguptaaa as a collaborator in the forked repo I am working on and mention it in the draft PR I have made. I will also contact @aryanguptaaa to share the bounty when we get that.
@devilkiller-ag let's talk and coordinate on discord ?
Thank you @andreamari
@devilkiller-ag let's talk and coordinate on discord ?
Here is my Discord ID: AshmitJaiSaritaGupta#5291
Issue Description
Add download option button to tutorials in docs, including
.ipynb
option. However, we would like to have the source code as it is, in terms of.md
files, without adding .ipynb files inmitiq/docs/source/examples/
, as Jupyter notebooks are hard to track in version control on Github. So, ideally, somehow the Jupyter notebooks should be created somehow (at documentation build or with another script), but still appear in the version of the docs that is hosted on readthedocs. It's a tricky issue :).Proposed Solution
Find solution in other sphinx documentations.
Additional References
See the top right button in this QuTiP notebook: https://nbviewer.org/urls/qutip.org/qutip-tutorials/tutorials-v4/time-evolution/003_qubit-dynamics.ipynb