Allow test setup/cleanup

[nstarman]

Thanks for making this great tool! In https://sybil.readthedocs.io/en/latest/patterns.html#migrating-from-sphinx-ext-doctest it says to migrate testsetup directives to invisible-code-block. I would like a parser that makes testsetup an alias for invisible-code-block, so that I can keep my testsetup blocks and stay compatible with sphinx.ext.doctest. Can such a parser be added to Sybil?

[cjw296]

What's your use case for keeping support for sphinx.ext.doctest?

[nstarman]

Primarily pytest-doctestplus, which uses most of the sphinx.ext.doctest syntax. It. would be easier to discuss docs with collaborators who are used to pytest-doctestplus if the testsetup <-> invisible-code-block mapping were done behind-the-scenes.

[cjw296]

My recommendation would be pick one tool and stick with it: if you want to use Sybil, then switch to it, if you want to use pytest-doctestplus, then use that.

Trying to support multiple example testing tools with subtly different semantics is not something I'd recommend, so while there's nothing stopping you doing what you suggest in your own code base, it's not something I'll land in a release.

[nstarman]

@cjw296, I got it working with

class ExtendedPythonCodeBlockParser(AbstractCodeBlockParser):

    def __init__(self, future_imports=()) -> None:
        super().__init__(
            [
                DirectiveLexer(directive=r"code-block"),
                DirectiveLexer(directive=r"testsetup"),
                DirectiveLexer(directive=r"testcleanup"),
                DirectiveInCommentLexer(directive=r"(invisible-)?code(-block)?"),
            ],
            "python", PythonEvaluator(future_imports)
        )

    pad = staticmethod(pad)

using this class to replace the standard PythonCodeBlockParser Sybil provides. While this works, it's somewhat inelegant. I tried instead making a custom DirectiveParser, using DocTestDirectiveParser as my template, but it didn't work.

class TestSetupDirectiveParser:

    def __init__(self, optionflags=0) -> None:
        self.lexer = DirectiveLexer(directive="testsetup")
        self.string_parser = DocTestStringParser(DocTestEvaluator(optionflags))

   def __call__(self, document: Document) -> Iterable[Region]:
        for lexed in self.lexer(document):
            source = lexed.lexemes["source"]
            for doctest_region in self.string_parser(source, document.path):
                doctest_region.adjust(lexed, source)
                yield doctest_region

My conftest looked like

pytest_collect_file = Sybil(
    parsers=[
        DocTestParser(optionflags=ELLIPSIS),
        TestSetupDirectiveParser(),
        PythonCodeBlockParser(),
        SkipParser(),
    ],
    patterns=["*.rst", "*.py"],
).pytest()

Do you have any recommendations to make the DocTestDirectiveParser-variant work?

[cjw296]

I'm surprised your ExtendedPythonCodeBlockParser works, given that your codeblocks have doctest rather than python in them judging by https://github.com/search?q=repo%3Aastropy%2Fastropy+testsetup&type=code?

As for TestSetupDirectiveParser, if you get it right, I don't think you'll need to override __call__.

[nstarman]

Thanks! I'll give it another shot. This is for https://cosmology-compat-astropy--15.org.readthedocs.build/en/15/.