Parse reference titles as reStructuredText

[chrisjsewell]

Feature Suggestion

Parse the reference title text with self.state.inline_text in the sphinx.ReferenceRole (the same approach used, for example, by admonition titles, rubric titles, table titles and parsed-literal role)

Description

In sphinx rST, if you write :ref:`*title* <ref>`, *title* is interpreted as literal text and translated to a literal node: https://github.com/sphinx-doc/sphinx/blob/697dff31ab09625ead62e1a7ec0780126aeb07c6/sphinx/roles.py#L77

This IMO is not ideal and there are many use cases where one would expect the text to be parsed as rST.

In MyST-Parser, I have overriden some of the reference processing code (see here), such that nested syntax is parsed as "expected". For example the following Markdown is parsed correctly by Markdown parsers:

[This is a title with nested __*bold italic*__ syntax](ref) 

This is a title with nested bold italic syntax

and now is also correctly parsed into docutils AST, before post-transforms:

<document source="root/index.md">
    <paragraph>        
        <pending_xref refdomain="True" refexplicit="True" reftarget="ref" reftype="myst" refwarn="True">
            <inline classes="xref myst">
                This is a title with nested 
                <strong>
                    <emphasis>
                        bold italic
                 syntax

and after post-transforms:

<document source="root/index.md">
    <paragraph>
        <reference internal="True" refid="ref">
            <inline classes="std std-ref">
                This is a title with nested 
                <strong>
                    <emphasis>
                        bold italic
                 syntax

If there is interest in this, I would consider spending some time writing up a PR to upstream some of these alterations.

[tk0miya]

Thank you for posting. But reStructuredText is not markdown. Basically, it does not allow nested inline mark-ups. So we should not break the concept of reST. -1 for accepting nested mark ups.

[chrisjsewell]

But reStructuredText is not markdown. Basically, it does not allow nested inline mark-ups.

Obviously its completely up to you guys. But I would point out that the lack of nested inline parsing in rST is an implementation limitation, not a conceptual design choice. In fact directly in the docutils code it states:

    def parse(self, text, lineno, memo, parent):
        # Needs to be refactored for nested inline markup.
        # Add nested_parse() method?

(https://github.com/live-clones/docutils/blob/34e25f023498658990cbdae91ec817aa2cd2234a/docutils/docutils/parsers/rst/states.py#L614-L616)

Also it is in the TODO notes: https://docutils.sourceforge.io/docs/dev/todo.html#nested-inline-markup99, and https://docutils.sourceforge.io/docs/dev/rst/alternatives.html#nested-inline-markup

[tk0miya]

Yes, as you commented, the spec of reStructuredText should be discussed in the docutils project. This is not good place for that. Sphinx is a mere user of reStructuredText format. We should not enhance it independently.

[chrisjsewell]

One last comment, in case it was not clear from my initial one.

I am not talking here about changing the rST spec, I'm suggesting parsing the reference title text with self.state.inline_text in the sphinx.ReferenceRole, an approach that is already used in a number of other docutils and sphinx roles, including admonition titles, rubric titles, table titles and the parsed-literal role.

[tk0miya]

AFAIK, there is no roles supporting nested inline parsing in both docutils and Sphinx. In my understanding, the concept of reStructuredText does not allow nested inline mark-up now. I agree that there is a plan to support it in the future of reST. But it isn't so at present. I'd not like to allow nested inline mark-ups in Sphinx until docutils allows.

[chrisjsewell]

👍 . Maybe "nested inline parsing" was a poor choice of words by me though, since is not exactly what I am proposing (although it would eventually be nice); it is only to parse the titles as rST. I have changed the title and original description of this issue to reflect that. If you still do not wish to allow this, that's ok (less work for me 😆) but I just wanted to make sure there was no miscommunication

[tk0miya]

I believe I did not misunderstand your comments. In my knowledge, there are no interpreted text (a.k.a. roles) that parses given title as an inline-marked-up text in docutils and Sphinx. So I'd not like to do that because it conflicts the concept of (current) reST spec. Please let me know if my memory is incorrect.

[chrisjsewell]

Well, to my knowledge, there are no other roles with "titles", there is just the examples I have mentioned of directives where the "title" is parsed as rST and other roles which parse their text as rST. But anyway, I feel like I have clarified the original intent for this issue now, so I don't want to push it any further.

[astrojuanlu]

This would be so cool to have. At the moment I am experimenting writing more content in MyST and was excited to see, as a user/writer, that it supports a wide range of nested markup. If there is a low-risk way of lifting this limitation, without breaking reST, that would be great.