Any hints as to how to override the OutputChecker

[matthew-brett]

We (over at https://github.com/nipy/nipy) have a customized doctest OutputChecker class for a few new directives - https://github.com/matthew-brett/nipy/blob/pytest/nipy/conftest.py#L152

I have tried to apply this output checker by monkey-patching doctest in conftest.py:

doctest.OutputChecker = NipyOutputChecker

I can see, with a debug test in a .py file, that shows me that, when I get to the doctest, doctest.OutputChecker is my checker.

https://github.com/matthew-brett/nipy/blob/pytest/nipy/algorithms/statistics/formula/formulae.py#L95

But the checkers check_output never gets called.

I wondered idly whether this is because y'all enforce your OutputChecker in DoctestModulePlus:

https://github.com/scientific-python/pytest-doctestplus/blob/main/pytest_doctestplus/plugin.py#L268

What is the best way to plumb in my OutputChecker, when using pytest-doctestplus?

[pllim]

You mean, you want to use your own custom doctest directives on top of what doctestplus provides?

[pllim]

I wonder if what you have can be merged into this package (unless they duplicate existing functionality). Though the sympy one is tricky unless you can make it optional (i.e., enabled only if sympy is installed).

[matthew-brett]

Yes - exactly - in my case I want to extend the options available to the doctest processing - and to do that - I need to change the OutputChecker - just for reference for anyone who doesn't knowdoctest - this stuff:

https://docs.python.org/3/library/doctest.html#doctest.DocTestRunner

I can certainly make a PR with the code we have, and make sympy optional - but it would be very good to have some extension mechanism such that you can plumb such things without modifying the doctestplus code.

[pllim]

Hmm, I am not expert in that, so I'll defer to @bsipocz and @saimn on this.

[matthew-brett]

I see that it would be moderately difficult to do - but it would be very useful.

[saimn]

Not sure if there is a good way to do it, given the way it's done currently :grimacing: I guess you should have a look at doctest's code to see why we need both https://github.com/scientific-python/pytest-doctestplus/blob/main/pytest_doctestplus/plugin.py#L268 and https://github.com/scientific-python/pytest-doctestplus/blob/main/pytest_doctestplus/plugin.py#L198 (or maybe we don't need both).

[ev-br]

Is this still relevant @matthew-brett ? I need to make DoctestParser pluggable in scipy-doctest, and started with OutputChecker since it's easier: https://github.com/ev-br/scpdt/pull/141

[matthew-brett]

I spent some time trying to weave my way into the depths of monkey-patching, before giving up, and accepting the constraints of doctest-plus for doctesting. So - yes - still relevant - but I don't have any great wisdom on how the output checking should best be extended here.

[ev-br]

Not sure how to do it with pytest-doctestplus either. Do you feel like giving an alternative a go? If you do, we can take it to the tracker https://github.com/ev-br/scpdt --- for context see https://github.com/scipy/scipy/pull/20127 and https://mail.python.org/archives/list/scipy-dev@python.org/thread/XH5RDJ5GZRRBP3UAQGWNWMH6P3KYZAUA/