Document `re.Scanner`

[JosephSBoyle]

Documentation

re.Scanner is currently marked as "experimental" but was added to the standard library 24 years ago[0]. People seem to think this is an interesting feature[1][2] and a github code search for re.Scanner( gives >500 hits[3], suggesting people have found practical uses for the class.

Proposal

I suggest we add a few lines to the docs describing what it is and perhaps a short docstring with an example.

Some previously collated discussions: https://github.com/python/cpython/issues/91308#issuecomment-1093947379

[0]. https://github.com/python/cpython/blob/dbefa88b27ee1f124f782363b139aee3f1ccf590/Lib/re/__init__.py#L327 [1]. https://news.ycombinator.com/item?id=36517749 [2]. https://www.reddit.com/r/ProgrammerTIL/comments/4tpt03/python_til_about_rescanner_which_is_useful_for/ [3]. https://github.com/search?q=re.Scanner%28+language%3APython&type=code

[AlexWaygood]

@vstinner put together a list of places where this has previously been discussed here: https://github.com/python/cpython/issues/91308#issuecomment-1093947379

Cc. @serhiy-storchaka as well, for interest

[rhettinger]

I believe that Scanner was intentionally left undocumented and only left in because someone might have found it and relied on it. The tool was deemed to be a dead end.

[serhiy-storchaka]

One of problems is that it may conflict with the name of internal class _sre.SRE_Scanner if we make it public like we did with _sre.SRE_Pattern and _sre.SRE_Match (now exposed as re.Pattern and re.Match for documentation purposes). _sre.SRE_Scanner is used in implementation of re.Pattern.finditer(), but it also the type of the result of undocumented re.Pattern.scanner(). If we decide to document re.Pattern.scanner(), we will have a problem.