Python API to convert .md to .html using Sphinx with sphinx_conf

[paugier]

Context

For applications for which we need to parse programmatically MyST and produce html (for example pelican-myst-reader or a potential MyST to slide converter), it is very useful to be able to use sphinx with particular sphinx extensions (myst_parser, sphinxcontrib.bibtex, sphinx_exercise, sphinx_subfigure to name a few).

From the current documentation, I don't see how it can be done. From https://myst-parser.readthedocs.io/en/v0.18.0/docutils.html, I guess one should write something like

from docutils.core import publish_string
from myst_parser.sphinx_ import Parser

source = "hallo world\n: Definition"
output = publish_string(
    source=source,
    writer_name="html5",
    settings_overrides={
        "myst_enable_extensions": ["deflist"],
        "embed_stylesheet": False,
    },
    parser=Parser(),
)

However, it is not clear how a sphinx_conf dictionary can be included.

Note that there was already a similar issue with the previous API: https://github.com/executablebooks/MyST-Parser/issues/327

Proposal

• It should be easy to use myst_parser programmatically with sphinx extensions.

• A small example should be included in the documentation (I guess in https://myst-parser.readthedocs.io/en/v0.18.0/api/reference.html).

Tasks and updates

No response

[chrisjsewell]

It should be easy to use myst_parser programmatically with sphinx extensions.

Hey @paugier I'm afraid this is just not really possible, since sphinx does not expose an API that one can really use outside the build of the entire project.

This is as opposed to docutils, which has this API that myst can build on, but obviously lacks sphinx features.

The previous API you mention kind of did this, but in a quite "hacky" way that was really only for testing and certainly not maintainable or for public use.

I feel, you/someone would need to push this change upstream in Sphinx to make it possible

[paugier]

Thanks @chrisjsewell for your explanation. It's really frustrating to see that the full power of MyST can't be used outside of Jupyterbook or a documentation using Sphinx. Solving this would open some nice perspectives.