quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.77k stars 309 forks source link

keep-ipynb notebook with cell magics and yaml options can't be executed #2123

Open jjallaire opened 2 years ago

jjallaire commented 2 years ago

e.g.

```{python}
#| output: asis
%%javascript

// some JS code


This is because cell magics need to be the first line of the cell. 

Note that the `keep-ipynb` feature is really designed for debugging rather than for producing a final notebook for distribution (e.g. markdown constructs like citations and crossrefs are not resolved in this version). We should either resolve this or make it more clear in the documentation that `keep-ipynb` is for debugging intermediate representations (we could even change the filename to make this more clear).
JanPalasek commented 2 years ago

I'd like to add that you encounter the same issue in JupyterLab if you develop with Quarto, which is not just an intermediate notebook. You just can't use cell magics with the "attributes" in any ipynb notebook.

jjallaire commented 2 years ago

One way to resolve this would be to accept cell options within cell metadata (so outside of source)

cscheid commented 2 years ago

We could also improve the cell option parsing code so that it knows to skip magics. It already is parameterized wrt language because of different comment syntax anyway...

jjallaire commented 2 years ago

There is another problem though: those option commands are actually invalid javascript (and Jupyter doesn't know to exclude them). I think the metadata solution is probably the only way.

cscheid commented 2 years ago

There's another reason that this isn't great syntax:

```{python}
#| output: asis
%%javascript

// some JS code


It confuses our vs code tooling since it takes that to be Python code.

It looks to me like we're stealing the `{python}` syntax here to mean "actually a jupyter cell with a python engine, but then do this other thing instead".

Couldn't we introduce new syntax that means, simply, a "jupyter raw cell" so that it can be converted correctly to .ipynb?
jjallaire commented 2 years ago

This will convert correctly to ipynb in that cell magics are indeed inside Python cells (Jupyter notebooks can only have one type of executable cell). "juptyer raw cell" has a different meaning which is a call that contains raw HTML, TeX, etc. We might happen to know that %%javascript means "just emit the script tag" but can't a priori know what all cell magics do so I'd rather just handle them at face value (also, we can't introduce new syntax for notebook editing, and I'd like to have the expression of this be the same across qmd and ipynb.

cscheid commented 2 years ago

EDIT: I take it back, that's not how jupyter works.