executablebooks / MyST-Parser

An extended commonmark compliant parser, with bridges to docutils/sphinx
https://myst-parser.readthedocs.io
MIT License
751 stars 197 forks source link

Better merging of eval-rst parsing result #953

Open chrisjsewell opened 2 months ago

chrisjsewell commented 2 months ago

Ideally, the result of parsing with eval-rst would better merge with the parent document, i.e. ensuring all the "internal variables" are merged like: https://github.com/live-clones/docutils/blob/266f359408130ca25f6f161453140e73f169408a/docutils/docutils/nodes.py#L1799

at the moment, if you add to tests/test_sphinx/sourcedirs/footnotes/footnote_md.md e.g.

```{eval-rst}
check eval-rst integration

[#f]_ [#]_ [*]_

.. [#f] restructuredtext footnote definition (f)
.. [#] restructuredtext footnote definition (#)
.. [*] restructuredtext footnote definition (*)

the footnotes will not be correctly recorded.

You need to add to `myst_parser/mdit_to_docutils/base.py` something like:

```python
def render_restructuredtext(self, token: SyntaxTreeNode) -> None:
    """Render the content of the token as restructuredtext."""
    # copy necessary elements (source, line no, env, reporter)
    newdoc = make_document()
    newdoc["source"] = self.document["source"]
    newdoc.settings = self.document.settings
    newdoc.reporter = self.reporter
    # pad the line numbers artificially so they offset with the fence block
    pseudosource = ("\n" * token_line(token)) + token.content
    # actually parse the rst into our document
    MockRSTParser().parse(pseudosource, newdoc)
    # TODO the merging of documents below is a bit hacky, and needs to be improved
    # (ideally this would also be upstreamed to docutils/sphinx)
    for node in newdoc:
        if node["names"]:
            self.document.note_explicit_target(node, node)
    self.document.footnotes += newdoc.footnotes
    for key, _nodes in newdoc.footnote_refs.items():
        self.document.footnote_refs.setdefault(key, []).extend(_nodes)
        for _node in _nodes:
            self.document.note_refname(_node)
    self.document.symbol_footnotes += newdoc.symbol_footnotes
    self.document.symbol_footnote_refs += newdoc.symbol_footnote_refs
    self.document.autofootnotes += newdoc.autofootnotes
    self.document.autofootnote_refs += newdoc.autofootnote_refs
    self.current_node.extend(newdoc.children)

BUT still needs extra stuff to merge ids etc