scoop: try to use py.exe to run pipx.pyz if python.exe is not found in PATH

[jborbely]

This is not an issue with pipx, but relates to how pipx is installed with scoop on Windows.

Describe the bug

Installing pipx with scoop works (i.e., pipx gets installed), but the issue is that the pre_install command explicitly uses python.exe to run pipx.pyz; however, some systems may be configured to use py.exe to run scripts.

This is an issue if python.exe is not in PATH

>python -V
'python' is not recognized as an internal or external command,
operable program or batch file.

because the system is configured to use the Python Launcher for Windows

>py -V
Python 3.12.2

How to reproduce Make sure python.exe is not in PATH and then run

>scoop install pipx
Installing 'pipx' (1.4.3) [64bit] from main bucket
Loading pipx.pyz from cache
Checking hash of pipx.pyz ... ok.
Running pre_install script...
Linking ~\scoop\apps\pipx\current => ~\scoop\apps\pipx\1.4.3
Creating shim for 'pipx'.
'pipx' (1.4.3) was installed successfully!
'pipx' suggests installing 'python'.

>pipx ensurepath
'python' is not recognized as an internal or external command,
operable program or batch file.

Expected behavior

Running scoop install pipx should create the pipx.bat script that supports using python.exe or py.exe to run pipx.pyz. The _preinstall command should check how the python interpreter is invoked in the terminal to decide how to run pipx.pyz, for example,

{

    "pre_install": [
      "$exe = 'python'",
      "if ((Get-Command python -ErrorAction SilentlyContinue) -eq $null) {",
      "  if (Get-Command py -ErrorAction SilentlyContinue) {",
      "    $exe = 'py'",
      "  }",
      "}",
      "$cmd = '@' + $exe + ' \"%~dp0pipx.pyz\" %*'",
      "Set-Content -Value $cmd -Path \"$dir\\pipx.bat\""
    ],

}

I'm not a PowerShell user, so there is most likely a more elegant way to decide the name of the python executable, but the above example illustrates what I'm trying to say.

One could also save the logic of which interpreter to use within pipx.bat.

If I manually modify the contents of ~\scoop\apps\pipx\1.4.3\pipx.bat to be @py "%~dp0pipx.pyz" %*, I can successfully run pipx

>pipx --version
1.4.3

[jborbely]

As a alternative to using the pre-install command in the scoop manifest to decide which python executable is written to the pipx.bat script, the batch script could decide at runtime which executable to use.

The search order could be

1. Use the PIPX_DEFAULT_PYTHON environment variable (this is how I originally tried to solve the 'python' is not recognized error)
2. Use python.exe
3. Use py.exe
4. If all fail, display a message indicating the 3 ways to solve the issue (with search priority)

[Gitznik]

Hi @jborbely, thanks for raising this. If you are interested in contributing a fix, you're welcome to open a PR.

[jborbely]

@Gitznik thanks for your reply and for the suggestion.

I considered opening a PR, but I couldn't find the scoop manifest file in the pipx repository so I didn't understand what I would edit and creating a new manifest from scratch didn't seem correct since one already exists. Maybe opening a PR at https://github.com/ScoopInstaller/Main would make sense to update the pipx manifest file in the bucket, but this seems inappropriate for someone who is not a maintainer of pipx to do.

Perhaps the simplest way forward is for the person who manages the pipx manifest file in scoop to decide if they want to copy-paste one of my two suggestions below into the manifest file to replace the existing "pre_install" value.

Option 1: Decide which executable to use when pipx is installed with scoop

    "pre_install": [
        "$exe = \"python\"",
        "if ((Get-Command python -ErrorAction SilentlyContinue) -eq $null) {",
        "  if (Get-Command py -ErrorAction SilentlyContinue) {",
        "    $exe = \"py\"",
        "  }",
        "}",
        "$cmd = '@' + $exe + ' \"%~dp0pipx.pyz\" %*'",
        "Set-Content -Value $cmd -Path \"$dir\\pipx.bat\""
    ],

Option 2: Decide which executable to use when pipx is run

    "pre_install": [
        "$src = @'",
        "@echo off",
        "set cmd=\"%~dp0pipx.pyz\" %*",
        "",
        "if defined PIPX_DEFAULT_PYTHON goto environ",
        "where /q python.exe && goto classic",
        "where /q py.exe && goto launcher",
        "",
        "echo pipx cannot find a Python interpreter.",
        "echo Perform one of the following actions, which are listed in pipx search priority:",
        "echo 1. Define a PIPX_DEFAULT_PYTHON environment variable with the full path to python.exe as the value",
        "echo 2. Add the directory where python.exe is located to the PATH environment variable",
        "echo 3. Add the directory where py.exe (Python Launcher for Windows) is located to the PATH environment variable",
        "goto done",
        "",
        ":environ",
        "%PIPX_DEFAULT_PYTHON% %cmd%",
        "goto done",
        "",
        ":classic",
        "python.exe %cmd%",
        "goto done",
        "",
        ":launcher",
        "py.exe %cmd%",
        "goto done",
        "",
        ":done",
        "",
        "'@",
        "Set-Content -Value $src -Path \"$dir\\pipx.bat\""
    ],

There is an overhead with Option 2 to execute each where call, e.g., on my system with 24 directories in PATH there is about a 200-ms overhead per where call. For example, to run pipx --version with Option 1 implemented takes about 0.7 seconds to complete whereas Option 2 takes about 1.1 seconds to determine that py.exe will be used to run pipx.pyz and then complete.

For me, it doesn't matter which Option (if any) the manifest maintainer chooses.

[uranusjr]

I wonder if we should just use py.exe and don’t deal with PATH at all. That’s why py.exe exists in the first place.

[jborbely]

That would suggest Option 1 is better, but the default is py instead of python and the if checks are adjusted accordingly.

I would still recommend "pre_install" to check whether to use py or python at install time since people may be using conda/mamba in which case they don't use py at all.

[Gitznik]

Or they may have installed python via the microsoft store, which also does not install py.exe AFAIK.

[jborbely]

@Gitznik you are correct that Microsoft Store does not install py when it installs Python. Also, if someone installs Python via the Microsoft Store the executable name is python3.minor. This means that there are at least 8 different cases of executable names that could potentially be available on a Windows PATH to run pipx.pyz depending on how Python was installed

• py -- python.org
• python -- python.org, conda, mamba, ...
• python3.minor -- Microsoft Store (currently, the Store can install minor values of 7, 8, 9, 10, 11 and 12)

Microsoft Store does include python.exe and python3.exe in a subdirectory to python3.x.exe when Python 3.x is installed, but, by default, the subdirectory containing these generic executable names is not in PATH.

[YUKI2eN3e]

What about using ftype Python.File to determine the executable to use? That way you would know if they are using py.exe, python.exe or whatever else for their main python install.

[jborbely]

@YUKI2eN3e Unfortunately, that may not be reliable on different computers and the availability of the ftype command depends on the terminal that is used.

For example, using the Command Prompt

C:\Users\username>ftype Python.File
File type 'Python.File' not found or no open command associated with it.

C:\Users\username>py -VV
Python 3.12.3 (tags/v3.12.3:f6650f9, Apr  9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)]

and using PowerShell

PS C:\Users\username> ftype Python.File
ftype : The term 'ftype' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ ftype Python.File
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (ftype:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

[YUKI2eN3e]

@jborbely How about making a "check-python-runner.bat" script sort of like this:

@echo off
for /f "tokens=2 delims==" %%a in ('assoc .py') do ftype %%a

and calling it like .\check-python-runner.bat so that it runs in cmd (which has assoc and ftype) regardless of if you are calling it from cmd or pwsh?

Or since in my case there are extra args:

> .\check-python-runner.bat
Python.File="C:\Windows\py.exe" "%L" %*

If you wanted just the executable you could make a "check-python-runner.ps1" something like this:

$AssociatedName = cmd.exe /c "assoc .py";
$FTypeRunner = cmd.exe /c "for /f `"tokens=2 delims==`" %a in ('assoc .py') do @ftype %a";
$PyRunner = $FTypeRunner.Substring($AssociatedName.length-2) -Split '"';
$PyRunner[0];

so that when I run it I get just the executable:

> pwsh .\check-python-runner.ps1 
C:\Windows\py.exe

[jborbely]

@YUKI2eN3e Your check-python-runner.ps1 script depends on how Python was installed. The .py file extension may not have an association.

For example, consider if someone is using conda. They would typically use an Anaconda Prompt to interact with Python

(base) PS C:\Users\username> conda --version
conda 24.3.0
(base) PS C:\Users\username> python --version
Python 3.12.2
(base) PS C:\Users\username> .\check-python-runner.ps1
File association not found for extension .py
File association not found for extension .py
You cannot call a method on a null-valued expression.
At C:\Users\username\check-python-runner.ps1:3 char:1
+ $PyRunner = $FTypeRunner.Substring($AssociatedName.length-2) -Split ' ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

Cannot index into a null array.
At C:\Users\username\check-python-runner.ps1:4 char:1
+ $PyRunner[0];
+ ~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

conda/mamba/pyenv-win modify the PATH of the shell to make python.exe available – no file associations are defined.

I still believe that using Get-Command in _preinstall of the scoop manifest (to determine the name of the python executable to use in pipx.bat when scoop install pipx is run) is the safest way forward.

Iterate over a list of possible executable names and pick the first one that works.

An example script that could be defined in _preinstall is

$exe = $null
$aliases = @('py', 'python', 'python3')
for ($minor=22; $minor -ge 7; $minor--) {
  $aliases += 'python3.{0}' -f $minor
}
foreach ($alias in $aliases) {
  if (Get-Command $alias -ErrorAction SilentlyContinue) {
    $exe = $alias
    break    
  }
}
if ($exe -eq $null) {
  throw [System.IO.FileNotFoundException] "Python executable not found. Please install Python before installing pipx or add the directory where the Python executable is located to the PATH environment variable."
}
$cmd = '@{0} "%~dp0pipx.pyz" %*' -f $exe
Set-Content -Value $cmd -Path "pipx.bat"

Note: the minor loop in this example starts at 22 and goes to 7, implying that this script may be okay for about 10 years and also supports EOL (3.7, soon 3.8, eventually 3.9, ...) versions. Perhaps the Python versions that may be installed via the MS Store should only be versions that the latest pipx supports, but this would require more book-keeping chores for the scoop-manifest maintainer by periodically editing the start-end values in the loop. Maybe a bot can do this, not sure (probably not worth the overhead).

Perhaps @sitiom (as the person who added pipx as a scoop bucket in https://github.com/ScoopInstaller/Main/pull/5315) has some suggestions on how to update bucket/pipx.json to resolve this issue.