search

quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.98k stars 328 forks source link

manuscript project not rendering when there is a labelled code chunck #11088

Open jacques-shebehe opened 1 month ago

jacques-shebehe commented 1 month ago

Bug description

A qmd document is well rendered with figures and tables labelled. But when the document is used as manuscript project, quarto fails to render the manuscript project if there one labelled code chunck. Jupyter kernel starts and carries out the computation in the code chunck but quarto fails to render if there is at least one labelled code chunk: I have tested *| label: fig-tagand *|tbl-tag.

Steps to reproduce

---
title: Quarto Manuscript Default Project
authors:
  - name: Norah Jones
    affiliation: The University
    roles: writing
    corresponding: true
bibliography: references.bib
jupyter: nbstata
---

## Section
This is a simple placeholder for the manuscript's main document [@knuth84].

The document fails to render once the below code chunk @fig-figuretag1 is labelled with `*| label: fig-figurelabel`.

```{stata}
*| label: fig-figuretag1
*| fig-cap: From code chunck
*| echo: false

qui webuse auto, clear
scatter price mpg

The same happens if there is one table code chunk labelle with *| label: tbl-tablelabel.

@fig-figuretag1 worked well from code chunk.

and @tbl-tabletag1 embedded after being produced in a the code chunk below.

*| include: false
table foreign, stat(mean price) stat(sd price)
collect layout (foreign) (result)
collect style cell result, font("Trade Gothic Next", bold size(12))
collect export tab1, as(html) replace

Even here we refer to the same html table @tbl-tabletag1

::: {#tbl-tabletag1}

*| echo: false
*| output: asis
display "```{=html}"
type tab1.html
display "```"

Produced dynamically from formatted html table

:::

*| echo: false
*| include: false
table foreign, stat(mean price) stat(sd price)
collect layout (foreign) (result)
*collect style cell result, font("Trade Gothic Next", bold size(12))*
collect export tab2, as(md) replace

And when @tbl-tabletag2 is produced from Markdown table

::: {#tbl-tabletag2}

*| echo: false
*| output: asis
type tab2.md

Produced dynamically from md table :::

The End

</code></pre>

### Expected behavior

The qmd document should render and add cross-reference in a single document as well as in manuscript project.

### Actual behavior

In a manuscript project the same document fails to render if there is at least one labelled code chunk, be it labelle with code chuck label: tbl-, label: fig- or fenced div.

When there is a `*| label: fig-` in the index.qmd, there produced error is:

```bash
Rendering qmd embeds [index.qmd]
Rendering output notebook [index.qmd]
Error running filter /Applications/quarto/share/filters/main.lua:
Block, list of Blocks, or compatible element expected, got table
        while retrieving function argument content
        while retrieving arguments for function Div
stack traceback:
        /Applications/quarto/share/filters/main.lua:22368: in field 'render'
        /Applications/quarto/share/filters/main.lua:1233: in local 'filter_fn'
        /Applications/quarto/share/filters/main.lua:635: in function </Applications/quarto/share/filters/main.lua:625>
        (...tail calls...)
        [C]: in ?
        [C]: in method 'walk'
        /Applications/quarto/share/filters/main.lua:557: in function </Applications/quarto/share/filters/main.lua:546>
        (...tail calls...)
        /Applications/quarto/share/filters/main.lua:1334: in local 'callback'
        /Applications/quarto/share/filters/main.lua:1352: in upvalue 'run_emulated_filter_chain'
        /Applications/quarto/share/filters/main.lua:1388: in function </Applications/quarto/share/filters/main.lua:1385>
stack traceback:
        /Applications/quarto/share/filters/main.lua:557: in function </Applications/quarto/share/filters/main.lua:546>
        (...tail calls...)
        /Applications/quarto/share/filters/main.lua:1334: in local 'callback'
        /Applications/quarto/share/filters/main.lua:1352: in upvalue 'run_emulated_filter_chain'
        /Applications/quarto/share/filters/main.lua:1388: in function </Applications/quarto/share/filters/main.lua:1385>

When using ::: {#tbl-tabletag1} in the index.qmd, the error is:

Rendering qmd embeds [index.qmd]
Error running filter /Applications/quarto/share/filters/main.lua:
/Applications/quarto/share/filters/main.lua:22378: attempt to index a nil value (field 'content')
stack traceback:
        /Applications/quarto/share/filters/main.lua:1233: in local 'filter_fn'
        /Applications/quarto/share/filters/main.lua:635: in function </Applications/quarto/share/filters/main.lua:625>
        (...tail calls...)
        [C]: in ?
        [C]: in method 'walk'
        /Applications/quarto/share/filters/main.lua:557: in function </Applications/quarto/share/filters/main.lua:546>
        (...tail calls...)
        /Applications/quarto/share/filters/main.lua:1334: in local 'callback'
        /Applications/quarto/share/filters/main.lua:1352: in upvalue 'run_emulated_filter_chain'
        /Applications/quarto/share/filters/main.lua:1388: in function </Applications/quarto/share/filters/main.lua:1385>
stack traceback:
        /Applications/quarto/share/filters/main.lua:557: in function </Applications/quarto/share/filters/main.lua:546>
        (...tail calls...)
        /Applications/quarto/share/filters/main.lua:1334: in local 'callback'
        /Applications/quarto/share/filters/main.lua:1352: in upvalue 'run_emulated_filter_chain'
        /Applications/quarto/share/filters/main.lua:1388: in function </Applications/quarto/share/filters/main.lua:1385>

Your environment

Quarto check output

(base) m1pro14@MacBookPro q_ms % quarto check
Quarto 1.5.57
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.5.57
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /Library/TeX/texbin
      Version: 2024

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.11.5 (Conda)
      Path: /Users/m1pro14/anaconda3/bin/python
      Jupyter: 5.3.0
      Kernels: nbstata, python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.4.1
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
      knitr: 1.48
      rmarkdown: 2.27

[✓] Checking Knitr engine render......OK
mcanouil commented 1 month ago

Could you properly format your post using code blocks for code and terminal outputs? Thanks. If your code contains code blocks, you need to enclose it using more backticks, i.e., usually four ````. See https://quarto.org/bug-reports.html#formatting-make-githubs-markdown-work-for-us.

Could you try without a proprietary software such as Stata? This seems to be an issue with the Jupyter kernel you are using rather than Quarto itself.

jacques-shebehe commented 1 month ago

Could you properly format your post using code blocks for code and terminal outputs? Thanks. If your code contains code blocks, you need to enclose it using more backticks, i.e., usually four ````. See https://quarto.org/bug-reports.html#formatting-make-githubs-markdown-work-for-us.

Could you try without a proprietary software such as Stata? This seems to be an issue with the Jupyter kernel you are using rather than Quarto itself.

Thanks. I hope the post is now properly formatted.

Stata alone does not yet support Quarto. Jupyter kernel nbstata allows Stata users to take advantage of Quarto's features. I have already discussed this issue with Tim Huegerich @hugetim who the developed nbstata. If it were a Jupyter kernel issue, the code should not only fail in a manuscript with a labeled code chunk, but also when the code chunk is unlabeled in a manuscript project and in a default quarto document.

This issue is more similar to this one #8976 previously reported for R.

mcanouil commented 1 month ago

Anyhow, we do not have Stata licence. If the issue is Jupyter as you claim it is, then using python (and proper kernel) instead of stata should lead to the same error.

jacques-shebehe commented 1 month ago

I mean the issue is probably a Quarto issue and not a Jupyter kernel issue. The kernel starts and finalize the computation, the problem occurs when Quarto renders the output to html. The same code is rendered correctly to html with cross-referenced "labeled" code chunks in a single quarto document, but it fails in a manuscript project. If the code chunks are not labeled (or cross-referenced) the code renders even in a manuscript project.

A similar behavior was reported here #8976. That's why I think it is a Quarto issue.

mcanouil commented 1 month ago

Still, there is no way we can reproduce because we don't have Stata, we need you to do the tests. Please test the same setup using Python and default kernel.

If you cannot reproduce with default kernel for Python when using Jupyter or using knitr and R as engine, then the issue is the kernel and what it produces.

Because a kernel finishes does not mean it did a correct job (according to Quarto's expectations).

cderv commented 1 month ago

The kernel starts and finalize the computation, the problem occurs when Quarto renders the output to html.

@jacques-shebehe thank you for your report!

Can you share the rendered ipynb file ? Perks of Jupyter engine is that Quarto won't require to execute the cells if we call quarto render doc.ipynb so it shouldn't require stata on our side, and we should be able to reproduce and fix by looking at the ipynb file.

So can you share the ipynb of your reproducible example please ? keep-ipynb: true may be enough. Otherwise, you can write your example as a .ipynb and not a .qmd and you should still reproduce by

This way we'll know this is a problem with Quarto not handling something, possibly related to what nbstata output in jupyter.

Thank you.

hugetim commented 1 month ago

@jacques-shebehe Could you also please provide the contents of the '_quarto.yml' file used for your manuscript project?

Also, in case it's helpful, our prior discussion is here: https://www.statalist.org/forums/forum/general-stata-discussion/general/1765340-literate-programming-and-cross-reference-of-formatted-tables

jacques-shebehe commented 1 month ago

Can you share the rendered ipynb file ? Perks of Jupyter engine is that Quarto won't require to execute the cells if we call quarto render doc.ipynb so it shouldn't require stata on our side, and we should be able to reproduce and fix by looking at the ipynb file.

So can you share the ipynb of your reproducible example please ? keep-ipynb: true may be enough. Otherwise, you can write your example as a .ipynb and not a .qmd and you should still reproduce by

  • Opening your ipynb file in Jupyter (lab or notebook)
  • Execute all the cells
  • Save the .ipynb file
  • call quarto render on the ipynb.

This way we'll know this is a problem with Quarto not handling something, possibly related to what nbstata output in jupyter.

Thank you.

@cderv

Using keep-ipynb: true in the _quarto.yml file of the manuscript project is not outputing a ipynb file. There was an index.embed.ipynb in the manuscript roof folder, but I was unable to attach it to this post.

Instead I have been able to reproduce the error using an ipynb file in JupyterLab:

  1. Opened index.ipynb in JupyterLab
  2. Restared Kernel and Run All cells.
  3. quarto render index.ipynb

As with the qmd file, the Kernel started and completed the computation in he ipynb document. But Quarto fails to render the document in a manuscript project.

The error as below:

(base) m1pro14@ ms_project % quarto render index.ipynb     
Rendering output notebook [index.ipynb]
Error running filter /Applications/quarto/share/filters/main.lua:
/Applications/quarto/share/filters/main.lua:22361: attempt to index a nil value (field 'content')
stack traceback:
        /Applications/quarto/share/filters/main.lua:1233: in local 'filter_fn'
        /Applications/quarto/share/filters/main.lua:635: in function </Applications/quarto/share/filters/main.lua:625>
        (...tail calls...)
        [C]: in ?
        [C]: in method 'walk'
        /Applications/quarto/share/filters/main.lua:557: in function </Applications/quarto/share/filters/main.lua:546>
        (...tail calls...)
        /Applications/quarto/share/filters/main.lua:1334: in local 'callback'
        /Applications/quarto/share/filters/main.lua:1352: in upvalue 'run_emulated_filter_chain'
        /Applications/quarto/share/filters/main.lua:1388: in function </Applications/quarto/share/filters/main.lua:1385>
stack traceback:
        /Applications/quarto/share/filters/main.lua:557: in function </Applications/quarto/share/filters/main.lua:546>
        (...tail calls...)
        /Applications/quarto/share/filters/main.lua:1334: in local 'callback'
        /Applications/quarto/share/filters/main.lua:1352: in upvalue 'run_emulated_filter_chain'
        /Applications/quarto/share/filters/main.lua:1388: in function </Applications/quarto/share/filters/main.lua:1385>
ERROR: Error
    at renderFiles (file:///Applications/quarto/bin/quarto.js:78081:29)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async renderProject (file:///Applications/quarto/bin/quarto.js:78479:25)
    at async Command.actionHandler (file:///Applications/quarto/bin/quarto.js:83077:32)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8017:13)
    at async Command.parseCommand (file:///Applications/quarto/bin/quarto.js:7907:20)
    at async quarto (file:///Applications/quarto/bin/quarto.js:118224:9)
    at async file:///Applications/quarto/bin/quarto.js:118244:9
    at async mainRunner (file:///Applications/quarto/bin/quarto.js:118128:9)
    at async file:///Applications/quarto/bin/quarto.js:118235:5

To reproduce this error, the _quarto.yml, index.ipynb, and all output files are accessible in this repo: https://github.com/jacques-shebehe/ms_project_ipynb.git

P.S: Quarto renders the doc in a single document, but fails when the document is the article of a manuscript project.

jacques-shebehe commented 1 month ago

@jacques-shebehe Could you also please provide the contents of the '_quarto.yml' file used for your manuscript project?

Also, in case it's helpful, our prior discussion is here: https://www.statalist.org/forums/forum/general-stata-discussion/general/1765340-literate-programming-and-cross-reference-of-formatted-tables

@hugetim

The _quarto.yml file is

project:
  type: manuscript

manuscript:
  article: index.qmd

format:
  html:
    comments:
      hypothesis: true
  docx: default
  jats: default

  # (other formats)
  # pdf: default

execute:
  freeze: true

The index.qmd and _quarto.yml files are accessible in this repo: https://github.com/jacques-shebehe/q-ms-project.git

Thank you!

mcanouil commented 1 month ago

I think we don't need Stata or any non default kernel.

See simpler reproducible example:

I would close this issue in favour of #11117 since I think the issue is the same.

jacques-shebehe commented 1 month ago

I think we don't need Stata or any non default kernel.

See simpler reproducible example:

I would close this issue in favour of #11117 since I think the issue is the same.

@mcanouil Thank you! I follow the issue [#11117] and hope its solution will solve the issue with nbstata kernel.