IDLE: make smart indent after comment line consistent

[terryjreedy]

BPO	32918
Nosy	@terryjreedy, @csabella
PRs	python/cpython#5755

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/terryjreedy' closed_at = None created_at = labels = ['3.8', 'expert-IDLE', 'type-bug', '3.7'] title = 'IDLE: make smart indent after comment line consistent' updated_at = user = 'https://github.com/terryjreedy' ``` bugs.python.org fields: ```python activity = actor = 'terry.reedy' assignee = 'terry.reedy' closed = False closed_date = None closer = None components = ['IDLE'] creation = creator = 'terry.reedy' dependencies = [] files = [] hgrepos = [] issue_num = 32918 keywords = ['patch'] message_count = 2.0 messages = ['312613', '312614'] nosy_count = 2.0 nosy_names = ['terry.reedy', 'cheryl.sabella'] pr_nums = ['5755'] priority = 'normal' resolution = None stage = 'test needed' status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue32918' versions = ['Python 3.6', 'Python 3.7', 'Python 3.8'] ```

[terryjreedy]

https://docs.python.org/3/reference/lexical_analysis.html#blank-lines says "A logical line that contains only spaces, tabs, formfeeds and possibly a comment, is ignored" but notes that REPLs might not ignore them during interactive input. The same is true of editors as lines are typed. In particular, even a no-code comment line can suggest where a smart indenter should indent. In any case, Python does not care what follows '#', and IDLE should not either (except when uncommenting-out lines).

Suppose one types the following:

if 1:
    if 2:
        print(3)
    #
    #x
    # x

Currently, IDLE indents 4 columns after '#' and '# x' and 8 columns after '#x'. If one moves the comments to the margin, the indents are 0 and 8 columns. This is because pyparse.Parser._study2 ignores lines that match _junkre and _junkre only matches comments with '#' followed by a non-space (/B) character.

I think that IDLE should generally assume that the comment starts on a legal column and that the comment will apply to the next line typed, which will have the same indent, and that the lack of space after '#' (there have been many such in idlelib) is preference, indifference, or error.

The only exception relevant to IDLE is '##' inserted at the beginning of code lines to (temporarily) make them ignored. If one places the cursor at the end of such a line and hits return to insert new lines, some indent is likely wanted if the line above is indented. Matching '##.*\n' is easy enough.

Note that smart indent always uses the line above, if there is one, because there typically is not one below, and if there is, either choice is arbitrary and being smarter would cost time. (This should be in revised doc.)

[terryjreedy]

So I am proposing _junkre = re.compile(r"(?:[ \t]|##.\)\n").match with test change to match.