Open renefritze opened 2 years ago
Hi @renefritze -- We have limited developer time so this this package is currently in a maintenance mode (at the moment) as we figure out the "best" way to support ipynb
as an output for the jupyter-book project. Having said that we (QuantEcon
) use this package daily to build notebooks
that support our download notebook feature
for our lecture sites such as: https://python.quantecon.org/intro.html
So I understand your workflow request pretty well I think as we have a very similar need.
The technical
downside of this package
is that it takes the input content (md:myst, ipynb) and then passes it through the sphinx abstract syntax tree
before it arrives at the writer
in this package to extract the information from that tree and write ipynb
again which is a lossy
process. We have worked hard to make that as lossless
as possible but as sphinx
internals change this writer/translator
package will have to be maintained etc.
There are some side projects in the works in the executable book
community exploring rendering of myst
in juypyter
using javascript
which might be able to render the directives
and roles
introduce by md:myst
. As you say you can use jupytext
to convert md:myst
to ipynb
but the roles and directives markup will remain unprocessed in the jupyter
notebook. If these could render we wouldn't necessarily need to transit the sphinx ast
However code-cell currently do not get executed/their outputs included
This is a feature we had in the original version of this package sphinxcontrib-jupyter but we removed it as it used nbclient
(after writing ipynb
) and not the jupyter-cache
so it doubles up on computation runs.
Regardless of where we end up with more direct
translation routes (i.e. sphinx agnostic), there is still a use case for this package for rst
based projects in sphinx
.
Does that give you a bit more information about this package? Happy to follow up and discuss ideas etc.
Thank you very much for all that background info. Yes, the current situation is now much clearer.
Executing the produced notebooks "again" was an idea I also had, and discarded just the same.
Since you see utility for this package, even after a potential "frontend myst" thingy becomes available, I guess I'd consider working on #5 worthwhile.
I might put in a little research time over the next couple of weeks of how the interaction with the jupyter cache works in myst-nb
and to see how it could be used here. I'm not super familiar with sphinx/docutils internals however. I've written tiny extensions, but nothing major.
thanks @renefritze -- your help is appreciated -- let me know how you get on.
I'm also not familiar with jupyter-cache
but I know who wrote it (great programmer -- so should be entry points we can use etc.) and we can try and get some pointers from them after we learn a bit more about it. I can also try and answer any questions.
The myst-parser
and myst-nb
layers have just gone though a massive re-write recently, with myst-nb in the final development phases.
https://github.com/executablebooks/MyST-NB/pull/380
The team is still looking at how to support the current glue
behaviour (across pages as the last hurdle) but I bring this up as there will be some internal changes on how the sphinx.ast
looks with regards to images (as outputs from cells) etc. as there will be no CellOutputNode
node type. So you might want to wait a week to see if that PR is merged first -- we will then be updating jupyter-book
to leverage the new myst-nb
version and I'll have better insight into this integration with jupyter-cache
also.
Alright, thanks for the heads up. I'll be watching https://github.com/executablebooks/MyST-NB/pull/380
Coming from here (after almost a year..) I'm somewhat confused about the current and planned state of the ecosystem and hope somebody can clarify this for me.
I have notebooks in myst markdown in a sphinx project. Currently the html builder processes those, I get html output with all the code-cells' output in it. Myst-nb also outputs
.ipynb
s for me. However those have unprocessed sphinx roles/directives in the markdown cells. To fix that I would usesphinx-tojupyter
, run itsjupyter
builder and get nice markdown cells in the output notebooks that would be (mostly?) equivalent to the text produced from thehtml
builder. However code-cell currently do not get executed/their outputs included. That would be the goal of #5. After that I would need some more glue code in my deployment that puts thesphinx-tojupyter
.ipynb
into the html structure so they can be downloaded from there when I upload my final page.Is that about right?
I've also stumbled upon this in #3:
This I don't understand at all. Would jupyter somehow be rendering sphinx directives itself?
I do have some resources to work on #5, if that's mostly what's needed for me to get where I want to be. However if that's not a long-term solution anyway, maybe the work would be better spent elsewhere?