Allow executing an installed binary

jaraco commented 1 year ago

In #72, this project added support for making installed binaries available on the PATH. That means it's now possible to do something like:

pip-run cowsay -- -c "import subprocess; subprocess.run(['cowsay', 'hello world'])"

This got me to thinking that it might be nice to allow a user to invoke one of those binaries directly. Since the current syntax only allows passing parameters to a Python interpreter, it's not readily possible to execute anything else.

I'm imagining a syntax to indicate to run a new command in a subprocess, something like -! (in place of --):

pip-run cowsay -! cowsay 'hello world'

There are some problems to consider:

Existing install

If the dependency supplying the binary is already satisfied by the python environment, pip-run will not install it, and if that binary isn't on PATH, it will fail to run. And if there's another binary earlier on the PATH, it will be run instead.

I don't have an elegant solution to this issue other than to accept it (YOLO).

Interpreter resolution

It's not obvious to me that a pip-installed python script will have the appropriate environment to run.

After some brief investigation, however, I'm not too concerned. It does look as if these executables get a shebang pointing to sys.executable and PYTHONPATH will indicate the needed paths, so it will work in many (most?) cases.

 $ pip-run cowsay
Python 3.11.3 (main, Apr  7 2023, 20:13:31) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import pathlib
>>> pathlib.Path(sys.path[1]).joinpath('bin', 'cowsay').read_text()
"#!/opt/homebrew/opt/python@3.11/bin/python3.11\n# -*- coding: utf-8 -*-\nimport re\nimport sys\nfrom cowsay.main import cli\nif __name__ == '__main__':\n    sys.argv[0] = re.sub(r'(-script\\.pyw|\\.exe)?$', '', sys.argv[0])\n    sys.exit(cli())\n"
>>> print(pathlib.Path(sys.path[1]).joinpath('bin', 'cowsay').read_text())
#!/opt/homebrew/opt/python@3.11/bin/python3.11
# -*- coding: utf-8 -*-
import re
import sys
from cowsay.main import cli
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(cli())

Syntax considerations

The -- separator is a common pattern used in tox and other command line tools to separate groups of CLI parameters. I'm not aware of another convention for different designators.

Maybe instead of -!, the -- could be re-used, but the indicated command could include some syntax to indicate that it's a subprocess, something like:

pip-run cowsay -- !cowsay 'hello world'

If using that approach, I'd want to have confidence that the prescribed syntax would likely never be a valid syntax to a Python interpreter.

jaraco commented 1 year ago

I found another case today where it would have been useful to run an arbitrary command. I wanted to pip-run jaraco.develop but then invoke a git command (whose mergetool was configured to use py -m jaraco.develop). If the pip-run syntax would have allowed executing an arbitrary binary, that would have been readily possible:

pip-run jaraco.develop -! git pull ...

jaraco commented 1 year ago

After further consideration, I'm now leaning strongly toward ! prefix, as that's used both by xonsh and IPython.

I thought I'd check to see what happens when passing a !-prefixed string to Python, and surprisingly, it seemed to be willing to accept it:

 ~ @ py -3.12 !foo
/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/jaraco/foo': [Errno 2] No such file or directory

Oh, I was confused. CPython was never receiving the leading !. My xonsh shell was stripping that. Python treats it as a literal string:

 cpython main @ py -3.12 '!foo'
/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/jaraco/code/python/cpython/!foo': [Errno 2] No such file or directory

So inferring a subprocess by a leading ! could interfere with someone who wanted to execute Python on a file whose name literally has a leading !. Shold there be an escape for that case? For now, I'm going to say YAGNI, and presume that ! is safe until I discover otherwise.

jaraco / pip-run