Allow HTML comments that are successfully parsed by Pandoc near code blocks

[fehrlich]

Bug description

With upgrading from 1.3 -> 1.4 (1.4.5.49), this error appeared, apparently from commented python blocks. Downgrading removed the error.

Steps to reproduce

---
title: "Bug"
format: html
---

<!--```{python}

``` -->

```{python}

all three blocks are required

### Expected behavior

Show one python block

### Actual behavior

Error:

Cell In[1], line 1

    ^
SyntaxError: invalid syntax

Your environment

IDE: vscode OS: ubuntu

Quarto check output

Quarto 1.4.549
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.11: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.549
      Path: /opt/quarto/bin

[✓] Checking tools....................OK
      Chromium: 869685
      TinyTeX: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/bin
      Version: 2021

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.0 (Conda)
      Path: /opt/miniconda/envs/paper/bin/python
      Jupyter: 5.7.1
      Kernels: python3

[✓] Checking Jupyter engine render....OK

R scripting front-end version 4.1.2 (2021-11-01)
[✓] Checking R installation...........(None)

      Unable to locate an installed version of R.
      Install R from https://cloud.r-project.org/

[mcanouil]

Thanks for the report. I am very surprised it ever worked to comment a code block like that. For me, the normal way to comment would be the following. It's readable and parsable.

I am not sure this qualifies as a bug/regression in my opinion, but let's see what @cscheid thinks.

---
title: "Bug"
format: html
---

<!--
```{python}

-->

[fehrlich]

Thanks for the report. I am very surprised it ever worked to comment a code block like that. For me, the normal way to comment would be the following. It's readable and parsable.

This is the default way of vscode to comment, when you toggle line comments for the block.

[cderv]

Regarding VSCODE behavior it seems there was some change and revert

• https://github.com/quarto-dev/quarto/issues/258

So the newlines after comment were removed.

Also commenting code block is not a way to have the block non execute

• https://github.com/quarto-dev/quarto/issues/347

Not sure what is the purpose of this syntax, but I would say it should not be used.

• Either eval: false and echo: false
• Or Use .content-hidden to hide the whole block in output (https://quarto.org/docs/authoring/conditional.html#content-hidden)

[cscheid]

This is another instance of confusion that's caused by our current lack of an actual grammar that describes the acceptable syntax for Markdown inputs. The HTML comment syntax does appear supported by Pandoc, but is not recognized by our tooling that detects cells to execute in Jupyter.

In the medium-to-long-term, we will switch to a unified parser for all of our Markdown handling, and then either this will be accepted or there will be a syntax error that explains it. Right now, we're in this unfortunate spot where it's not clear for users when their syntax will be correct.

In terms of what I think we should accept, I agree with Christophe as far as it describes the current situation. With that said, I think Quarto should support comments, and the HTML comment syntax seems to be parsed robustly by Pandoc:

$ cat 8506.qmd
---
title: "Bug"
format: html
---

<!--```{python}

``` -->

```{python}

```%
$ quarto pandoc -f markdown -t native 8506.qmd
[ RawBlock
    (Format "html") "<!--```{python}\n```\n```{python}\n``` -->"
, Para [ Str "```{python}" ]
, Para [ Str "```" ]
]

So Quarto could adopt the convention that an HTML RawBlock starting with <!-- and ending with --> is treated as a comment everywhere in our code, including controlling code execution.

This would require some structural changes on our side, but they are changes that we need to do anyway in the medium-to-long-run.