simplistix / sybil

Automated testing for the examples in your documentation.
https://sybil.readthedocs.io/en/latest/
Other
74 stars 14 forks source link

Support an alternative to invisible-code-block for doctests #16

Closed Gallaecio closed 5 years ago

Gallaecio commented 5 years ago

It took me a while to realize that invisible-code-block only works with code blocks.

As far as I can tell, it is currently not possible to write invisible code as context for a documentation example that has lines of output.

cjw296 commented 5 years ago

Can you give a simple example or two of what you're looking to do and what your proposed solution would look like?

Gallaecio commented 5 years ago

I tried to do this:

.. invisible-code-block: python
    remember_me = b'see how namespaces work?'

>>> remember_me
b'see how namespaces work?'

or

.. invisible-code-block: python
    remember_me = b'see how namespaces work?'

::

    >>> remember_me
    b'see how namespaces work?'

But I am told that “remember_me” is undefined, because it seems to work only for code-block, which does not allow to perform output check, as far as I know.

cjw296 commented 5 years ago

I'll looks like you're missing the whitespace line after .. invisible-code-block, here's an example of invisible code blocks and doctest sections interacting on the same namespace:

https://github.com/Simplistix/testfixtures/blob/master/docs/comparing.txt#L440-L464

What test runner are you using? pytest -v shows what blocks are being picked up, if your invisible code block isn't there, it's not correctly formatted.

Gallaecio commented 5 years ago

It works indeed. Thanks!!!

Gallaecio commented 5 years ago

For the record, I believe the underlying issue was not actually Sybil, but pytest using regular doctest. I was accidentally combining both type of tests, Sybil’s were picking up the hidden code, but doctest’s was not, and I though it was Sybil as well.

cjw296 commented 5 years ago

Oh yes, now that you mention it, I remember this and it's a pretty frustating bug in pytest: https://github.com/pytest-dev/pytest/issues/4954#issuecomment-478053157 Please follow up there in the hope it might get some priority :-)

cjw296 commented 3 years ago

Following up on this a long while later, adding addopts = -p no:doctest to your pytest config gets its doctest parser out of the way.