make *.rst files in Doc/ parseable by doctest

[cjerdonek]

BPO	15939
Nosy	@ezio-melotti, @cjerdonek
Files	issue-15939-1.patch issue15939-ctypes.diff: Patch to fix doctests in ctypes against 3.2 issue15939-ctypes-2.diff: Patch to fix doctests in ctypes against 3.4.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-feature', 'docs'] title = 'make *.rst files in Doc/ parseable by doctest' updated_at = user = 'https://github.com/cjerdonek' ``` bugs.python.org fields: ```python activity = actor = 'ezio.melotti' assignee = 'docs@python' closed = False closed_date = None closer = None components = ['Documentation'] creation = creator = 'chris.jerdonek' dependencies = [] files = ['27186', '27529', '31339'] hgrepos = [] issue_num = 15939 keywords = ['patch'] message_count = 8.0 messages = ['170441', '170444', '170791', '172634', '172642', '172664', '172669', '172671'] nosy_count = 3.0 nosy_names = ['ezio.melotti', 'chris.jerdonek', 'docs@python'] pr_nums = [] priority = 'normal' resolution = None stage = 'patch review' status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue15939' versions = ['Python 2.7', 'Python 3.3', 'Python 3.4'] ```

[cjerdonek]

Currently, when trying to parse the *.rst files in the Doc/ folder (i.e. not actually running them but simply generating unittest.TestCase instances from them by passing them to doctest.DocFileSuite()), five files yield errors.

This issue is to make it so that all of the *.rst files in our Doc/ folder are at least parseable by doctest without error.

I will submit a patch.

Below are the errors that currently occur:

(1) test fdoc:library/ctypes.rst crashed -- Traceback (most recent call last):
  ...
ValueError: line 55 of the doctest for ctypes.rst has an invalid option: '+WINDOWS'

(2) test fdoc:library/doctest.rst crashed -- Traceback (most recent call last):
  ...
ValueError: line 1592 of the docstring for doctest.rst has inconsistent leading whitespace: '      """))'

(3) test fdoc:library/multiprocessing.rst crashed -- Traceback (most recent call last):
  ...
ValueError: line 1016 of the docstring for multiprocessing.rst lacks blank after >>>: '>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>'

(4) test fdoc:whatsnew/2.4.rst crashed -- Traceback (most recent call last):
  ...
ValueError: line 1422 of the docstring for 2.4.rst lacks blank after >>>: '   >>>"""'

(5) test fdoc:whatsnew/2.7.rst crashed -- Traceback (most recent call last):
  ...
ValueError: line 1531 of the docstring for 2.7.rst lacks blank after   .: '      ...'

[cjerdonek]

The attached patch addresses all but Doc/library/ctypes.rst.

See the following python-dev e-mail for a question about how best to handle that case:

http://mail.python.org/pipermail/python-dev/2012-September/121721.html

[cjerdonek]

My opinion on the "+LINUX" and "+WINDOWS" doctest directives in Doc/library/ctypes.rst is simply to leave them alone for now. We can make those errors go away in code when running doctest by adding no-op calls to doctest.register_optionflag() as below:

+doctest.register_optionflag('WINDOWS') +doctest.register_optionflag('LINUX')

prior to constructing the doctest TestSuites.

[ezio-melotti]

If they don't work I see no reasons to leave them there. Either there's another way to run those tests on selected platforms that can be used instead, of if there's no way the tests should just be skipped.

[ezio-melotti]

The attached patch fixes the doctests in Doc/library/ctypes.rst.

[cjerdonek]

Either there's another way to run those tests on selected platforms that can be used instead

A way to do it could be to register new doctest directives that skip the marked test based on the value of sys.platform. If we remove the directives, we lose that information and won't be able to do it anymore.

[ezio-melotti]

A way to do it could be to register new doctest directives that skip the marked test based on the value of sys.platform.

That can be done for 3.4+ only though.

If we remove the directives, we lose that information and won't be able to do it anymore.

It actually seems quite easy to distinguish them. At the beginning 'libc' is created either using cdll.msvcrt (on windows) or CDLL("libc.so.6") (on Linux). There are examples that use windll and other Windows- or Linux-specific functions later on but it's usually quite evident. If we get lost we can always dig in the history or just run the tests and see where they fail.

FTR there were a few actual errors that I fixed (mostly about bytes vs string). I also removed unnecessary prints and added a few doctest directives to make some of the tests pass. 'from ctypes import *' is used quite frequently, and it would be better to change it, but I left it unchanged for now. I also removed an obsolete warning about the doctest directives in the example because Sphinx hides them.

[cjerdonek]

FTR there were a few actual errors that I fixed (mostly about bytes vs string).

Great! Note that for the others, I was just trying to make the files parseable rather than passing. Currently, doctest aborts out on the files I fixed without even attempting to run the examples.

That can be done for 3.4+ only though.

Well, yes and no. If there is a helper script to check documentation, it could be run on any branch regardless of whether the script is in source control. For example, I was planning to write such a script (from the regrtest patch I put together before) to help me when checking people's patches to code snippets in the documentation.

[AA-Turner]

We now run make doctest through CI. I think therefore this is fixed?

A