Open maximlt opened 7 months ago
Jupytext supports multiple "notebooks as markdown" formats. In the first post I used the MyST format, that IIUC exists to make it easier to work with MyST-NB (and preserve tags like hide-output
?). {code-cell}
looks like a directive but in practice in MyST code it isn't one, it's used to collect all cells and "turn them into a notebook". It might not be such a good idea to rely on it? OTOH we use MyST-NB to build the docs so relying on this could give us extra benefits, i.e. convert all notebooks to Markdown, even those that are not meant to use the Pyodide directive (Datashader).
There's also the Jupytext notebook format that looks more conventional. Notebook tags, can be declared as such, we could use that to declare that this is code fence must be processes with the pyodide directive (first internally converting the code fence to a myst directive). An advantage of this approach is that it's closer to raw markdown (some renderers ignore whatever comes after python
so highlight the code correctly, verified with the md preview in VS Code):
```python tags=["pyodide"]
param = 5
Still some analysis to do! Some question to answer too:
- How's the experience using JupyText?
- How much do we want to use it? I.e. would we be ready to convert all notebooks to that format?
The explored workflow is meant to work best in Jupyterlab with Jupytext and Jupyterlab-MyST. VSCode doesn't yet have the same capabilities (built-in or via extensions), i.e. opening a markdown file as a notebook (well, Quarto does that but we're not using it). Refs: https://github.com/mwouts/jupytext/issues/143, https://github.com/microsoft/vscode-jupyter/issues/1240
Conclusion: I would not be the one pushing this as its not clear to me if Jupytext will help us in the long run. And I'm afraid the workflow will never be great in VS Code.
.md
to .ipynb
format and back.I believe there are other solutions like quarto with Panel plugin and mkdocs with pyodide extension that are worth exploring before switching to jupytext because they might solve more problems.
If jupytext
does not get in the way of solving the rest of docs issues, then its ok for me to introduce. But it will require someone to do the work. And I don't get enough benefit to be pushing this across Panel or HoloViz.
It might be an idea to support requirements
in the meta data
And create python scripts that can convert
.md
file to a single pyodide .html
file for preview. It does not matter its the full sphinx rendered .html file. Something simpler will also help you test if pyodide works. This will be very valuable.If this is implemented then my workflow would be improved and I would look more favorably at jupytext.
Can this
kernelspec
can get in the way. With pip
that I use, its not that obvious how to control that name.Convert your .md
to a rendered notebook and/ or html file.
jupytext --to ipynb --execute doc/playground/pyodide/jupytext.md
jupyter nbconvert --execute --to html doc/playground/pyodide/jupytext.ipynb
See also https://discord.com/channels/1075331058024861767/1076148201326907472/1187350817905311794.
git clone https://github.com/holoviz-dev/nbsite.git
cd nbsite
git checkout experiment_directive_jupytext
cd site
python -m venv .venv
source .venv/Scripts/activate # windows git bash
pip install -r requirements.txt
Start jupyter lab
to render a notebook.
With quarto you can create a document like
With an output of
I.e. you have two, live streamlit applications in the document.
There is a dedicated VS Code extension enabling you to run the code
There is a preview with live reload.
The quarto CLI is very fast. It can also render jupyter notebooks. Even with live reload quarto preview document.ipynb
.
Its just light years ahead of jupytext.
https://github.com/holoviz-dev/nbsite/assets/42288570/2fe31be1-6ca3-4dca-83ae-f62090c9d548
I would like a similar experience using some Python/ Jupyter tool chain.
Thanks for your feedback Marc, it's very useful, I'll look more into it after the holidays. My quick feedback is that I think you're mixing too many things. My goal was trying to see whether it was possible to have all our files as Markdown files without changing our stack drastically and build new custom stuff. It seems what you are exploring is far far beyond the scope of what I've done.
Its just light years ahead of jupytext.
That seems a little unfair, the goal behind Quarto and Jupytext is very different, I've been amazed with the capabilities offered by Jupytext.
Warning! This is a hacky experimentation, no serious code review please π
The goal is to manage to seamlessly combine:
In this PR I have quickly hacked something that seems to work well but needs more experimentation:
{code-cell} ipython3
. I have added thejupytext.md
file in the example site.{code-cell}
to{code-block}
.CodeBlock
directive, and overrides it when Sphinx runs. Code blocks ({code-block} python
) will be treated normally, those that have thepyodide
flag will be treated exactly as the pyodide directive implemented by Philipp.What is missing is a way to declare that all the
{code-cell}
of a markdown file should have the pyodide flag. This could probably be declared in the jupytext header and parsed by the pyodide extension. Right now the flag has to be added manually in the markdown file.With this approach the markdown files look something like that (it's likely the header can be reduced, I haven't really tried, and yes the
:pyodide: null
is ugly, ignore that it can be improved for sure):