feature: doctest support

[malthe]

Did you check the tree-sitter docs?

• [x] I have read all the tree-sitter docs if it relates to using the parser

Is your feature request related to a problem? Please describe.

Doctests are typically embedded inside docstrings in Python code, e.g.

def factorial(n):
    """Return the factorial of n, an exact integer >= 0.

    >>> [factorial(n) for n in range(6)]
    [1, 1, 2, 6, 24, 120]
    """

Today, the grammar does not parse docstrings and is oblivious to the doctests potentially lurking inside.

Describe the solution you'd like

The grammar should parse these doctests so that we can have proper editing support.

Describe alternatives you've considered

No alternatives have been considered.

Additional context

Some more discussion on the topic:

• https://github.com/microsoft/pylance-release/discussions/4196
• https://www.reddit.com/r/neovim/comments/zbmqcc/is_it_possible_to_have_rust_doc_test_comments/

[amaanq]

This would be better suited for injections, so you can then filter out the actual docstrings with queries, and inject "pydoc" into them. Take a look at jsdoc or luadoc for inspirations if you'd like to embark on that! But nothing is actionable here for that, so I'll close this out

[malthe]

Documentation on language injections: https://tree-sitter.github.io/tree-sitter/syntax-highlighting#language-injection.