IDLE - deprecate running without a subprocess

[f52c13af-a5a0-4c40-a7a2-6ce855459ec8]

BPO	16123
Nosy	@rhettinger, @terryjreedy, @kbkaiser, @amauryfa, @larryhastings, @ned-deily, @serwy, @asvetlov, @ideasman42
Files	idle_deprecate.patch idle_deprecate_rev1.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['expert-IDLE', 'type-feature'] title = 'IDLE - deprecate running without a subprocess' updated_at = user = 'https://github.com/serwy' ``` bugs.python.org fields: ```python activity = actor = 'terry.reedy' assignee = 'none' closed = True closed_date = closer = 'terry.reedy' components = ['IDLE'] creation = creator = 'roger.serwy' dependencies = [] files = ['27405', '27437'] hgrepos = [] issue_num = 16123 keywords = ['patch'] message_count = 26.0 messages = ['171911', '171948', '172060', '172109', '172115', '172117', '172119', '176954', '176967', '176968', '176972', '182629', '182630', '182676', '182711', '182712', '184995', '196024', '196057', '213716', '213727', '222957', '249045', '249067', '255213', '370838'] nosy_count = 12.0 nosy_names = ['rhettinger', 'terry.reedy', 'kbk', 'amaury.forgeotdarc', 'larry', 'ned.deily', 'roger.serwy', 'asvetlov', 'ideasman42', 'python-dev', 'Ramchandra Apte', 'Torgil Svensson'] pr_nums = [] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue16123' versions = ['Python 3.6'] ```

[f52c13af-a5a0-4c40-a7a2-6ce855459ec8]

Removing the option of running IDLE without a subprocess would simplify the maintenance of IDLE. This should be deprecated for 3.4 and then fully removed in 3.5.

Here's a link to a brief discussion on IDLE-dev: http://mail.python.org/pipermail/idle-dev/2012-June/003124.html

Attached is an initial patch to issue a deprecation message.

[918f67d7-4fec-4a8d-93e3-6530aeb1e57e]

Make it pop up a dialog with the message too in the patch, please.

[ned-deily]

A popup menu on every invocation of IDLE would be a very user-unfriendly thing to do. If it's possible that a print to stderr might not be visible to a user, another solution might be to use the approach in Lib/idlelib/macosxSupport.py tkVersionWarning which causes the warning message to show up in the PyShell window.

[f52c13af-a5a0-4c40-a7a2-6ce855459ec8]

I agree that a message within the shell would be more informative to the user. The _rev1 patch also adds a message to the shell.

[asvetlov]

Looks good for me.

[1762cc99-3127-4a62-9baf-30c3d0f51ef7]

New changeset 0430986a8c03 by Andrew Svetlov in branch 'default': Issue bpo-16123: IDLE - deprecate running without a subprocess. http://hg.python.org/cpython/rev/0430986a8c03

[asvetlov]

Change priority to deferred blocker for reminding to remove no-subprocess mode in 3.5.

Thanks, Roger.

[f8a59035-e116-44bb-8c5c-ddaf44c84ed6]

Hi, I just found a use for this feature, with use_subprocess being optional its possible to load idle into a running python session.

-- import idlelib import idlelib.PyShell idlelib.PyShell.main()

This allows you to load idle inside an application that embeds python, is there some other way to do this without this option?

[f52c13af-a5a0-4c40-a7a2-6ce855459ec8]

Campbell, your example runs IDLE *with* a subprocess. The "use_subprocess" flag defaults to True.

[f8a59035-e116-44bb-8c5c-ddaf44c84ed6]

@Roger, I should have mentioned I had to add "-n" to the argv so subprocess would be disabled.

[asvetlov]

I think that usage is not officially supported, so we cannot grant this way will work forever.

[rhettinger]

FWIW, I help a lot of people with IDLE problems and sometimes the -n option is the only thing that saves them. ISTM that removing it is not such as a good idea.

[f52c13af-a5a0-4c40-a7a2-6ce855459ec8]

@Raymond: What is so broken about IDLE that requires using "-n" to fix it? Let's try to fix those problems rather than keeping up the maintenance burden of supporting two execution modes.

[rhettinger]

The issue is usually with firewalls, security software, socket issues, etc. While the root problem lies outside IDLE, often the simplest way to get someone up and running is to use the -n option.

This is one of many annoyances that arise when teaching students Python using IDLE:
* Preferences window crashing
* Default font-size on a retina mac is tiny
* Inability to shutoff the prominent warning messages
* Control-A goes to the beginning of the line, past the >>> prompt.
* On Windows, IDLE sometimes has a two second delay before it runs scripts
* Students find that IDLE mysteriously pastes a clipboard into the interactive prompt unintentionally
* It is a common complaint that IDLE hangs
* Getting the correct Tcl/Tk setup on Macs is problematic.
* Starting IDLE from the command line emits messages about "setCanCycle is deprecated"

[918f67d7-4fec-4a8d-93e3-6530aeb1e57e]

On 22 February 2013 22:56, Raymond Hettinger \report@bugs.python.org\ wrote:

Raymond Hettinger added the comment:

The issue is usually with firewalls, security software, socket issues, etc. While the root problem lies outside IDLE, often the simplest way to get someone up and running is to use the -n option.

This is one of many annoyances that arise when teaching students Python using IDLE:

• Preferences window crashing
• Default font-size on a retina mac is tiny
• Inability to shutoff the prominent warning messages
• Control-A goes to the beginning of the line, past the >>> prompt.
• On Windows, IDLE sometimes has a two second delay before it runs scripts
• Students find that IDLE mysteriously pastes a clipboard into the interactive prompt unintentionally
• It is a common complaint that IDLE hangs
• Getting the correct Tcl/Tk setup on Macs is problematic.
• Starting IDLE from the command line emits messages about "setCanCycle is deprecated"

----------

---

Python tracker \report@bugs.python.org\ \http://bugs.python.org/issue16123\

---

Not experienced any of the problems on Linux.

[terryjreedy]

Roger has submitted patches for some of the items on your list (off topic for this issue), especially crashes and stalls. Some still need review and commits. Mac tends to be the most difficult platform to get tests on.

While these would be better discussed on other issues or on idle-sig (mirrored on gmane), I will briefly comment on a few.

Cntl-A selects the entire window on Windows, while Home sends the cursor to the beginning of the line. On the current command line, it first goes after the prompt, and alternates before and after with repetition. This is kbk's design. On old command lines, it only goes before, and that is a bug to be fixed. I think there is an issue, but cannot find it.

I have never seen unprovoked pasting, nor an unexpected startup delay when running from the editor.

I cannot find 'setCanCycle' in 2.7 or 3.3 Lib/idlelib/*.py and 'deprecated' only appears once, in a comment. I expanded the 'setCanCycle' to the entire cpython repository and still did not find it, so that is a complete mystery.

[amauryfa]

The issue is usually with firewalls, security software, socket issues, etc

Surely nowadays there are better ways than sockets to communicate between processes? Pipes?

[terryjreedy]

Slight correction: While I do not believe I have seen *clipboard contents pasted, after running a program from the editor that prints (to the shell), I *have seen the last line reprinted when trying to enter something new as the prompt. This sometimes might look like pasting from the clipboard. I sometimes have to hit enter a couple of times and ignore the SyntaxError to get a clean prompt. I have no idea if this is exclusive to running in a subprocess; it should be fixed independently of this issue.

The idea of using pipes is being explored on Idle-sig.

[terryjreedy]

I opened bpo-18823 "Idle: use pipes instead of sockets to talk with user subprocess" I think this or a socket fix is needed before we remove -n.

[larryhastings]

If the 3.4 changes are done, can we either close this or remove 3.4 from the versions?

[terryjreedy]

Every enhancement issue can be bumped to 3.5.

[terryjreedy]

It would be nice if we are able to to this in 3.5, because the dependecy is accomplished, but not doing so is not a release blocker.

[614aff26-3714-425a-8e88-433a8d17f219]

I frequently use IDLE as an interactive shell to Python with the Editor attached. It's very nice for this purpose as it's lightweight and embedded in Python delivery. This is only possible with -n flag as without it each press on F5 will restart the interpreter and I lose all variables, graphs etc.

Under 2.7 (not related to this thread) I'm also unable to use interactive graphs [ ion()] in matplotlib without the -n flag.

Is there another way to run the module without restarting interpreter without the -n option?

Will 2.7 be affected by this change in the future?

[terryjreedy]

Torgil, thank you for the interesting feedback. I will respond to your four paragraphs slightly out of order.

1. Starting Idle with '-n', I verified that running 'print(a)' from the editor works after entering 'a=1' in the shell. The intended purpose of the editor is to edit files that run as is. From this viewpoint, the carryover behavior is a bug. Idle's original single process mode, now accessed with -n, only remains because communicating with a subprocess via sockets turns out to be a bit unreliable.

2. Using an editor as a extension of the shell is an alternate use. It does not have to be tied to -n. I have been thinking about alternate run modes and added 'run as input' to my list. A useful version might be this: don't require a name for the editor buffer; don't restart; enter each statement at a prompt and run as if typed. The latter would make all the code run in Shell visible and add each statement to the history list.

3. I am a bit puzzled by your matlab comment. When tkinter code is run from editor in a separate process, it successfully grabs keyboard and mouse input. (One of the reasons for the shift from 1 to 2 processes was to disentangle user tkinter guis from the Idle tkinter gui.) On the other hand, running import msvcrt; k = msvcrt.kbhit() on Windows does not work with either 1 or 2 processes because the Idle gui retains input focus.

If you want to pursue this, please open a new issue on the tracker or post to the idle-sig list or its news.gmane.org mirror gmane.comp.python.idle. In either case, say more about what 'not work' means.

4.This issue was opened before PEP-434 and before 2.7 maintenance was extended from 5 years (expiring now as 3.5 is released) to 10 years. Hence the deprecation notice was not added to 2.7. I do not expect this to change.

[terryjreedy]

I have some different ideas for this issue that would not necessarily involve removing -n. While -n is can be an inferior experience for users (user code blocks the GUI, other interference, especially tkinter), the reason given for deprecation is to simplify IDLE maintenance by eliminating alternate code paths.

Summary: a new rpc_local module might allow us to simplify code now, without removing -n, and provide a path to switching subprocess communication from sockets to pipes.

1. Simplify maintenance by instead isolating -n code in a new 'rpc_local' module that is imported 'as rpc' when -n is given. (A 'run_local' module might also be needed.) The goal would be to have the rest of IDLE as oblivious as possible as to where user code is executed. Other modules would send messages via rpc and not know whether they go to another process (currently via a socket) or or back to the main thread in the same process. I don't know how far this is possible. It would certainly involve some refactoring.

2. Make rpc_local less of a dummy by executing user code in a separate thread connected by a pair of Queues (and call it rpc_thread). I believe this would solve the problem of user code freezing IDLE. On the other hand, it might make user code importing tkinter worse. If so, I would consider declaring that unsupported and not run code containing 'tkinter'. Perhaps I am overlooking some important reason a thread is not already used.

3. If rpc_thread worked, multiprocessing could be used to turn it into a new rpc remote process module that communicated with pipes instead of socket ports.

[terryjreedy]

Deprecation has been done and a message is printed under the splash screen. With 5 more years of maintenance experience under deprecation, I have not experienced -n mode as a maintenance burden. So I have no inclination at present to remove it (or even implement any of the ideas above).

Deprecation usually means 'no maintenance'. For idlelib, that means no new tests specifically for -n mode (and there never were any). I don't do manual tests either. (More patches and more testing in regular mode is more important to me.) It is possible that some feature has been disabled in that mode, but there are no such reports and I have not encountered anything in my occasional experiments in that mode. (Such as when someone claim that something only worked with -n.)

I have occasionally touched blocks with 'if subprocess ... else ...' but it has not been hard to leave the else part alone.

A year or so ago, I added a 'Startup failure' section to the IDLE doc. It list multiple possible causes and things to try. -n is irrelevant to most of them. If someone knows of another current issue, open a new tracker issue.