Simulate command-line arguments for program run in IDLE

[39d85a87-36ea-41b2-b2bb-2be43abb500e]

BPO	5680
Nosy	@rhettinger, @terryjreedy, @taleinat, @mcepl, @asvetlov, @roseman, @wm75, @mlouielu, @csabella, @veganaiZe, @miss-islington
PRs	python/cpython#1589 python/cpython#13763 python/cpython#14184 python/cpython#14185
Files	idle-args.diff issue5680-patch2.diff: issue5680-patch2 cmd-arg.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/terryjreedy' closed_at = created_at = labels = ['3.8', 'expert-IDLE', 'type-feature', '3.7'] title = 'Simulate command-line arguments for program run in IDLE' updated_at = user = 'https://bugs.python.org/mrabarnett' ``` bugs.python.org fields: ```python activity = actor = 'miss-islington' assignee = 'terry.reedy' closed = True closed_date = closer = 'terry.reedy' components = ['IDLE'] creation = creator = 'mrabarnett' dependencies = [] files = ['13600', '16139', '34568'] hgrepos = [] issue_num = 5680 keywords = ['patch'] message_count = 27.0 messages = ['85341', '98864', '111838', '214484', '228269', '238729', '238734', '241206', '255209', '255215', '296491', '296499', '296602', '344213', '344247', '344248', '344249', '344251', '344252', '344348', '344351', '344382', '344487', '345942', '345943', '345946', '345947'] nosy_count = 15.0 nosy_names = ['rhettinger', 'terry.reedy', 'ggenellina', 'taleinat', 'gpolo', 'mcepl', 'mrabarnett', 'asvetlov', 'markroseman', 'wolma', 'Saimadhav.Heblikar', 'louielu', 'cheryl.sabella', 'veganaiZe', 'miss-islington'] pr_nums = ['1589', '13763', '14184', '14185'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue5680' versions = ['Python 3.7', 'Python 3.8'] ```

[39d85a87-36ea-41b2-b2bb-2be43abb500e]

Patch idle-args.diff adds a dialog for entering command-line arguments for a script from within IDLE itself.

[1fd7a44c-f7f2-43ed-9c9f-bafa512b8598]

A different patch to solve the same issue. This one uses a standard tkSimpleDialog to prompt for the command line, and follows the directives found at the top of the source (only took 8 years to implement... not so bad :) )

XXX GvR Redesign this interface (yet again) as follows:

• Present a dialog box for ``Run Module''

• Allow specify command line arguments in the dialog box

[83d2e70e-e599-4a04-b820-3814bbdb9bef]

Putting tjr and tal on nosy list cos it's IDLE. Apologies if I've got it wrong.

[22200024-de1a-4081-ad85-2ac04e6b54d2]

Submitting a patch for 3.4.

1. Allows the user to set command line arguments via a dialog box
2. Parsing - Maps the string which user entered into a list i.e. the same results are produced as when run via terminal/command prompt. The parsing methods returns a list, which is similar to the sys.argv, when you run script from command line.
3. Adds tests for the parsing method
4. Extends mock_idle for point 3

I am on Debian and don't have access to a windows machine. please try to run the tests on windows machine and see if its OK.(cmd handles arguments rather differently.)

Also, should we persist the command line argument that user entered across sessions?

[terryjreedy]

I have a recent idea that there should be a separate and persistent execution process for each file run. This would make it easy to persist the cmd lines args for a particular module.

[terryjreedy]

I closed bpo-23665 in favor of this one. Raymond's take: msg238108

[rhettinger]

Here's my comment from the duplicate tracker item: --------------------------------------------------

A number of IDEs support menu options to set the execution environment for programs under development and testing. In particular, it would be nice if IDLE let the user set command line arguments to be passed into sys.argv when running a script by pressing F5.

Here are some existing implementations for reference:

• Wing-IDE: https://wingware.com/doc/intro/tutorial-debugging-launch
• Visual Studio: https://www.youtube.com/watch?v=IgbQCRHKV-Y
• PyCharm: https://www.jetbrains.com/pycharm/help/run-debug-configuration-python.html

This feature will help users interactively develop and test command-line tools while retaining all the nice features of the IDE. I would personally find it useful when teaching students about how sys.argv works.

I suggest adding an option under the Run menu for Set Command Line arguments. It would trigger a dialog box that lets a user set a string such as "somefile.txt -r 10". The user string would be run through shlex to break it into separate fields. When F5 is pressed, the sys.argv list would be repopulated to include the script name and the lexed arguments.

A little more elaborate option is to add a Run menu entry for Set Execution Enviroment that let's the user 1) set the command-line 2) specify the start-up directory (using os.chdir), 3) and edit the environment variables (from os.environ) or at least be able to set PYTHONPATH.

[terryjreedy]

Pending application of a patch, the following will work to only add args to sys.argv when running from an Idle editor.

import sys
# ...
if __name__ == '__main__':
    if 'idlelib.PyShell' in sys.modules:
        sys.argv.extend(('a', '-2'))  # add your argments here.
    print(sys.argv)  # in use, parse sys.argv after extending it
    # ['C:\\Programs\\python34\\tem.py', 'a', '-2']

[terryjreedy]

Today on SO: https://stackoverflow.com/questions/33866724/read-data-from-file-idle. Person writing script to open a file given on command line asks 'How to test from IDLE?' He found PyCharm, but I would like to add a better answer than the workaround I gave above. It does not have to be either perfect or necessarily permanent. There are two subissues.

1) Get a command line from the user. 1a. What dialog? 1b. Retain it for as least the current session? Same for all editors? Specific to file?

2) Use the command line. 2a. parse with shlex or custom parser. 2b integrate in ScriptBinding.py when starting user code so code sees arguments in sys.argv.

Looking *briefly* at the patches:

M.B.: Get command line with new extension module argumentDialog.py. Parse with re.findall(arg_regex). Modifies ScriptBinding, including the deletin of a condition for setting user sys.argv.

G.G.: Get command line with tkSimpleDialog._QueryString. Split with shlex. Reuses MB's ScriptBinding patch.

S.H.: Get command line with custom dialog. Use custom parser. Add new tests.

My inclination is to start as simple as possible. Use a message box to get a single string (no error checking is needed). Set for specific file only, not all. The setting could be added to status bar. Parse with shlex. Question: is basic command line parsing system specific? Does MAC stick with unix/posix rules? Should shlex.split use non-posix mode on Windows?

[terryjreedy]

When file is run with command line args, it might be nice to add an additional separator line with the line entered. That way, if someone ran a series of experiments with different command lines, they would have the command line and output together as documentation that can be saved.

[terryjreedy]

I closed bpo-28889, with a 4th patch, in favor of this one.

Gabriel, you somehow never signed the PSF Contributor License Agreement. Until you do, I will assume that anything you did is covered by the other 3 patches.

[terryjreedy]

Running with arguments is only one requested or possible deviation from the standard F5 mode. See bpo-19042, msg296498, for many more. I am thinking that there should be one menu entry (Custom Run, Alt-F5)? that brings up a box with alternatives, initially defaulting to the F5 values, then persisting.

The order of execution should be a) Syntax check by compile, b) Run options, if requested, c) Restart Shell, if requested, d) Execute code object.

I want to start with setting sys.args, but with a mind to adding other things. Running without explicit saving might be next.

[d009eb67-2b4f-4621-8997-8c3e222ae3f4]

I've created a simple work-around for those who don't want to wait for the next release(s) of IDLE. It's available on SO:

https://stackoverflow.com/a/44687632/5039027

[rhettinger]

Can be bump this up in priority? The patch has been awaiting review for a good while.

[terryjreedy]

3 recent 'doit' requests (2 on PR 1589) raise my priority for this issue.

When a user runs a file from an editor, IDLE simulates, as well as it can, the user entering 'python -i path' in a system command line shell. Both on a command line and in IDLE, the program starts with sys.argv = ['path'] (at least on my Windows system). This can be shown by running a file with 'import sys; print(sys.path)' both ways.

The request for this issue is to simulate 'python -i path a1, a2, ...'. The program should start with sys.argv = ['path', 'a1', 'a2', '...'], with the argument strings being the same as they would be if entered at a command line on the *same system*.

I have looked at least a bit at all 4 patches and have decided to start fresh, using just a few existing lines. The latest of the 4 was submitted 2 years ago, while I was learning to use git.

By the following September, using config-extensions was obsolete. *If* we want to save a command string across sessions, a line could be added to config-main. But I am dubious about this. I expect that users will want to vary argument values for 1 program and need different sets of arguments for different programs.

I want to use a subclass of idlelib.query.Query for the query box. No need to reinvent it.

I believe that argument strings should be parsed into an argument list using shlex.split. No need to reinvent that either. But I am not an expert in this area and the doc it rather vague. I suspect that 'posix=False' should be used on Windows. Does anyone know? Gabriel's patch is the only one using shlex.split, but always with the default 'posix=True'.

Saimadhav tested his patch on Debian. The others did not say. A verified-by-human htest should be added to runscript.py, and manual test running 'import sys; print(sys.argv)' from an editor done on the 3 major OSes.

[csabella]

I'll can work on a PR for this, unless you want to do it based on your research.

[terryjreedy]

When running IDLE normally, with a subprocess, it would be possible to pass arguments to python itself when IDLE restarts the user process (with shell.restart_shell()). That would be another issue. We would have to look at whether any options would disable idlelib.run and how running code supervised might change results.

[terryjreedy]

My suspicion about posix=False was wrong.

>>> shlex.split(''' 1 "'a' 'b'" ''', posix=False)
['1', '"\'a\' \'b\'"']  # len(...[1]) = 9
>>> shlex.split(''' 1 '"a" "b"' ''')
['1', '"a" "b"']  # len = 7
f:\dev\3x>py -c "import sys; print(sys.argv)" 1 "'a' 'b'"
['-c', '1', "'a' 'b'"]  # Matches 'True'

>>> shlex.split(''' 1 '"a" "b"' ''', posix=False)
['1', '\'"a" "b"\'']
>>> shlex.split(''' 1 '"a" "b"' ''')
['1', '"a" "b"']  # Similar difference
f:\dev\3x>py -c "import sys; print(sys.argv)" 1 '"a" "b"'
['-c', '1', "'a", "b'"]  # crazy inversion and 3 args, matches neither

[terryjreedy]

Cheryl, go ahead after reading the notes above. Use an f-string for the runcommand argument and remember that it is run within the pristine user environment. Otherwise, let's start with minimal changes.

[csabella]

I've added a minimal change for running a module in the editor with command line arguments.

Some notes:

1. The shortcut key (F7) can be entered while on the shell window. As of now, any of the Run menu shortcuts can be entered, so I didn't fix that bug as part of this issue.
2. I'm not saving the arguments in any way.
3. I didn't (yet) make the change to print the command line arguments in the shell or to add it to the status bar.
4. I didn't make a change to get the sys.argv values from startup.
5. I added a single menu entry and not a cascade of different custom run options.

[rhettinger]

Thanks. Those seem reasonable for a first pass.

[terryjreedy]

Two design issues:

1. As I said 2017-06-20, msg296499, I expect to add more alternatives to the standard run to this menu option and dialog. (Cheryl, I should have pointed you to this post as well as the later ones.) I want to add just one new option to have only 1 new key to learn. I already cannot remember all of them and I expect I am not the only one. They need to be together on one dialog so that uses can conveniently pick multiple options when that makes sense. Running in a console with args is completely sensible. And exclusionary logic, such as 'Cannot supply arguments if no restart' should be easier on one dialog.

The configuration and pseudoevent names, perhaps 'run-custom', should be changed to reflect that now. For the menu, I am less sure. Perhaps 'Run Custom', 'Run Setup', 'Run with Alternatives', 'Alternate Runs'?

2. I am pretty sure I prefer shift-F5 for the key. (But cannot be sure until I try it.) Easier to learn I think.

[csabella]

I've pushed a change for a more generic dialog for a 'Run Custom' option and I changed the keybinding to Shift-F5. Sorry for not doing that on the initial commit; I didn't fully understand the difference between what you were asking for and the minimal version I had committed.

For this change, I also added a 'Restart shell' checkbox to the dialog, more as a proof-of-concept than anything else (although I did hook it up, so it should be functional). Is this the type of dialog you had in mind?

I made some comments on the PR as to some of the decisions I've run into so far with the design.

Thanks!

[terryjreedy]

New changeset 201bc2d18b60adb05810d2a6ab396047bc527088 by Terry Jan Reedy (Cheryl Sabella) in branch 'master': bpo-5680: IDLE: Customize running a module (GH-13763) https://github.com/python/cpython/commit/201bc2d18b60adb05810d2a6ab396047bc527088

[terryjreedy]

A more than minimal patch has been merged. Further work will be on new issues. Remaining problems not previously discussed:

1. Tab focus traversal is wrong. This is generic to Query subclasses that add widgets. See bpo-37325.

2. When the customize box is cancelled, focus goes to the Shell, even when I add 'event.widget.focus_set/force()'. When the parent of the box was the editor, 'self.shell.text.focus_set' did not work to shift it to the Shell text.

Does anyone else have any idea how to put the focus back on the editor after cancel?

[miss-islington]

New changeset 81f7899f517f18a45ab111598f9dd6d7210956a8 by Miss Islington (bot) in branch '3.7': bpo-5680: IDLE: Customize running a module (GH-13763) https://github.com/python/cpython/commit/81f7899f517f18a45ab111598f9dd6d7210956a8

[miss-islington]

New changeset ae526ee320d3feabba0aa4dffa9d52e39f8941dc by Miss Islington (bot) in branch '3.8': bpo-5680: IDLE: Customize running a module (GH-13763) https://github.com/python/cpython/commit/ae526ee320d3feabba0aa4dffa9d52e39f8941dc