0.19 broke `eval-rst` directives with colon fences and curly braces

[Yoshanuikabundi]

Describe the bug

context When I try to create an eval-rst directive with colon fences and curly braces, I get an error:

conf.py::

myst_enable_extensions = [
    "colon_fence",
]

index.md:

:::{eval-rst}
Some `RST <https://docutils.sourceforge.io/rst.html>`_
:::

expectation I expected the document to build successfully.

bug But instead I get this error message:

$ make html
docs/index.md:30: WARNING: Unknown directive type: 'eval-rst' [myst.directive_unknown]

problem

This is a very confusing error as the directive exists, it's just this particular combination fails. The same combination works with other directives:

:::{warning}
hi there
:::

Reproduce the bug

Note that using backtick fences:

```{eval-rst}
Some `RST <https://docutils.sourceforge.io/rst.html>`_

or omitting the curly braces:

```md
:::eval-rst
Some `RST <https://docutils.sourceforge.io/rst.html>`_
:::

works as expected; only the combination fails.

List your environment

I'm not using jupyter-book at all, but running MyST Parser through Sphinx. I confirmed this behavior on MyST Parser v0.19.1 and v1.0.0, but the bug does not exist on v0.18.1. I haven't tested on other versions (which do not exist on Conda Forge). The bug is present on Sphinx 5.3.0 and 6.1.3. An example Conda environment that produces the issue is:

conda list

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
alabaster                 0.7.13             pyhd8ed1ab_0    conda-forge
autodoc-pydantic          1.8.0              pyhd8ed1ab_0    conda-forge
babel                     2.12.1             pyhd8ed1ab_1    conda-forge
beautifulsoup4            4.12.2                   pypi_0    pypi
brotlipy                  0.7.0                    pypi_0    pypi
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2022.12.7            ha878542_0    conda-forge
certifi                   2022.12.7          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1                   pypi_0    pypi
charset-normalizer        3.1.0              pyhd8ed1ab_0    conda-forge
click                     8.1.3           unix_pyhd8ed1ab_2    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
cryptography              40.0.2                   pypi_0    pypi
css-html-js-minify        2.5.5                    pypi_0    pypi
docutils                  0.19                     pypi_0    pypi
idna                      3.4                pyhd8ed1ab_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        6.4.1              pyha770c72_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.2.0              h65d4601_19    conda-forge
libgomp                   12.2.0              h65d4601_19    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libsass                   0.22.0                   pypi_0    pypi
libsqlite                 3.40.0               h753d276_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libzlib                   1.2.13               h166bdaf_4    conda-forge
lxml                      4.9.2                    pypi_0    pypi
markdown-it-py            2.2.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.2                    pypi_0    pypi
mdit-py-plugins           0.3.5              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
myst-parser               1.0.0              pyhd8ed1ab_0    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
openff-sphinx-theme       0.0.32+224.gdb11c41          pypi_0    pypi
openssl                   3.1.0                h0b41bf4_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pip                       23.1               pyhd8ed1ab_0    conda-forge
projectname               0.0.0+0.gfe8b7b2.dirty          pypi_0    pypi
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydantic                  1.10.7                   pypi_0    pypi
pygments                  2.15.0             pyhd8ed1ab_0    conda-forge
pyopenssl                 23.1.1             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.11.3          h2755cc3_0_cpython    conda-forge
python-slugify            8.0.1                    pypi_0    pypi
python_abi                3.11                    3_cp311    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0                      pypi_0    pypi
readline                  8.2                  h8228510_1    conda-forge
requests                  2.28.2             pyhd8ed1ab_1    conda-forge
setuptools                67.6.1             pyhd8ed1ab_0    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.4.1                    pypi_0    pypi
sphinx                    6.1.3              pyhd8ed1ab_0    conda-forge
sphinx-click              4.4.0              pyhd8ed1ab_0    conda-forge
sphinx-notfound-page      0.8.3              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   1.0.4              pyhd8ed1ab_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.1              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_2    conda-forge
text-unidecode            1.3                      pypi_0    pypi
tk                        8.6.12               h27826a3_0    conda-forge
typing-extensions         4.5.0                hd8ed1ab_0    conda-forge
typing_extensions         4.5.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
unidecode                 1.3.6                    pypi_0    pypi
urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zipp                      3.15.0             pyhd8ed1ab_0    conda-forge

My OS is up-to-date Arch.

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

[chrisjsewell]

Heya, this is to some extent intended; colon fences are principally intended for "nested markdown" content, whereas backtick fences are for non-markdown content. Is there a reason not to use backtick fences?

[Yoshanuikabundi]

It's really nice to be able to syntactically see the difference between a code block and a directive - both as someone reading a lot of markdown and especially for tooling. With colon code fences, I can write and use markdown code block lints and not worry about getting wires crossed. If I could, I'd turn off backtick directives altogether so that each feature has its own syntax. It's also a lot of churn to update, and it's wierd to have to remember this for only one directive.

The docs seem to describe the feature as being for this purpose, and don't mention anything about it being for nested markdown in any way that doesn't also apply to backticks: https://myst-parser.readthedocs.io/en/latest/syntax/optional.html#code-fences-using-colons

[raphaelquast]

Since I just ran into the same problem, I thought I'll add my 2 cents here because I grew quite fond of MyST recently :-)

In my case, there was no particular reason why I used ::: instead of ```, I just expected both to behave similar. This issue was the only soruce of information that clarified that colon-fence syntax is not supported for {eval-rst}, (since the error-message did not help to pinpoint why things went wrong)

As I see it, your question "Is there a reason not to use backtick fences?" can also be reversed... is there a reason why you consider it a bad idea to allow using :::{eval-rst} (aside of enforcing syntax-conventions)?

From your docs I expected that ::: is practically a synonym for ``` and can be used for any directive.

By adding "colon_fence" to myst_enable_extensions (in the sphinx conf.py configuration file), you can also use ::: delimiters to denote directives, instead of ```.

[MacqGit]

As I ran into this issue as well, I just wanted to give you a quick "hands-up" as well. I'm namely responsible for managing all the documentation produced within the company I work for. The issue here is that we have lots of different documentation projects currently running which are maintained by my colleagues. As the system is already up for quite some time now, most of them have indifferently using the ... or ::: for declaring markdown or 'non-markdown' content (namely eval-rst). This summer I decided to upgrade our compilation infrastructure (via containers) to bring them to the last version of Sphinx (we where still running a rather old 5.X.X version of Sphinx) as well as the whole environment, including Myst-Parser. And that's where I ran into troubles because it forces me not only to "update" all the users about the change but also to adapt all the sources in order to get everything back to normal (meaning errorless compilations)...