Windows python3 executable

[ptyork]

ISSUE:

On Windows, a Python 3 install directory contains python.exe. However, there is no python3.exe, python3.cmd or similar. Thus, a user following almost any online tutorial, copy/pasting installation scripts, etc. will fail because all are written for *nix platforms that standardize on python3 as the name of the executable.

Worse, there is a python3.exe in a default AppData\Local\Microsoft\WindowsApps directory which is a stub to open the Windows Store to download Python 3. So the unsuspecting user is greeted with the option to install Python 3 from the Windows Store, which could break an existing installation and certainly will not do what they intended.

This is obviously only an issue for novice users. But I teach novice users. And this comes up numerous times each semester despite posting FAQ's and warnings related to this issue. And it seems very easy to fix in the default install.

This seems like it should be an issue long since reported, but I cannot find it by searching. So I apologize if this is a duplicate.

REQUESTED FIX:

One option is simply to add a copy of python.exe named python3.exe in the base install directory. This appears to be how PIP is handled, with pip.exe, pip3.exe and pip3.X.exe all being copies of the same executable located in a Scripts subdirectory.

Alternatively, adding a python3.cmd script that calls python.exe and forwards all args would serve the same function. Though I'm unsure that saving 100MB would be worth the potential confusion.

Linked PRs

• gh-100184

[FFY00]

@zooba do you have any proposals for this? On unix, we have symlinks up to the executable with the minor version: python > python3 > python3.12. I assume this is not the case on Windows because symlinks aren't reliably supported, so it would require us to copy the interpreter. Do you know if there is any other mechanism that allow us to implement this reliably? If not, would it be reasonable to install very simple launcher executables that mimic the symlink behavior? I definitely think this is a goal worth pursuing.

One option is simply to add a copy of python.exe named python3.exe in the base install directory. This appears to be how PIP is handled, with pip.exe, pip3.exe and pip3.X.exe all being copies of the same executable located in a Scripts subdirectory.

pip's case is a bit different, all the executables are launchers for a Python module.

[pochmann]

a user following almost any online tutorial, copy/pasting installation scripts, etc. will fail because all are written for *nix platforms that standardize on python3 as the name of the executable.

Hmm, I checked about 10 of the top Google results for python tutorial, and that's not at all what I saw...

[ptyork]

Thanks for the reply and attention.

pip's case is a bit different, all the executables are launchers for a Python module.

True, though I believe on Windows, python.exe simply invokes functions in python3X.dll. Here's a directory listing:

08/30/2021  07:36 PM           101,608 python.exe
08/30/2021  07:36 PM            59,624 python3.dll
08/30/2021  07:36 PM         4,488,424 python39.dll
08/30/2021  07:36 PM         2,504,187 python39.zip

The executable stubs are even smaller than the pip ones:

11/03/2021  08:44 AM           106,337 pip.exe
11/03/2021  08:44 AM           106,337 pip3.9.exe
11/03/2021  08:44 AM           106,337 pip3.exe

Yes, symbolic links exist on Windows, but only for NTFS, not FAT32. IMO, perhaps just following pip's lead and copying the executable stubs might be the lowest friction solution.

Hmm, I checked about 10 of the top Google results for python tutorial, and that's not at all what I saw...

Well, the first result...python.org...currently uses python3.11. How about I sidestep this and just modify my original assertion and say that "much of the documentation I send my students to" uses python3. But even if it uses python, my Mac users will have issues.

The REAL problem is that there is inconsistency between platforms and documentation. python works by default on Windows but python3 or python3.x does not. python does not work by default on MacOS or Linux (Debian/Ubuntu anyway), but python3 and python3.x does. I sent students to the MariaDB and MySQL connector pages. Windows folks had issues running python3 from MariaDB docs and Mac users had issues with python in the MySQL docs.

Likewise, pip works by default on Windows but not on Mac/nix, which need pip3; but many package installation guides give command line examples using pip and not pip3. And then there's the py launcher for Windows, which is cool but doesn't exist on Mac/nix and is unfortunately referenced in a number of tutorials. And I think the /usr/bin/python symbolic link is apparently supposed to be managed using update-alternatives on Linux, which I guess is similar, but is not installed by default nor is it available on Mac.

Anyway, point is there is inconsistency across platforms. And while some inconsistency is unavoidable, these seem unnecessary and do cause confusion for students...and distress for teachers. If we can just get python, python3, python3.X, pip, pip3, and pip3.x all working by default across all major platforms, that would be fantastic. Short of this, at least the _3 and _3.x ones...

...I guess asking for a default /usr/bin/python and /usr/bin/pip links to be added for Mac/*nix is probably out of scope for the issue, huh? ;)

[pochmann]

100MB

You mean 100 kB.

The python.org tutorial explains what to use for different systems, showing python3.11, py and python. Does your own tutorial also do that (in the intro chapter, not just in the FAQ) but students just choose to ignore that? (Maybe include a note that asking about it will cost them points in their next exam :-)

[ptyork]

Pedantry aside, yes my CS majors must of course learn the intricacies, though you'd be surprised at how little motivation many show to learn even the basics of terminal operations. I can and will hold these students accountable as it is a learning outcome. HOWEVER, students in other majors take classes or are simply given ancillary assignments in Python to provide exposure and understanding (and recruiting), not to master it. They copy/paste blocks of code that they find online or that I write. Having to constantly provide "translations" and then having them still encounter roadblocks and frustrations as they have to seek and wait potentially hours for assistance due to the need to add or remove a '3'...is avoidable pain. Pain that can needlessly drive away potentially talented technology majors, which is counter to our mission.

This is adding little to this issue. java/javac, dotnet, node/npm, gcc, clang, go, etc., etc. all are CLIs that are at least semantically identical cross-platform. There seems little reason for Python to be the outlier, especially as it is billed as an approachable introduction to coding.

[FFY00]

True, though I believe on Windows, python.exe simply invokes functions in python3X.dll. Here's a directory listing:

AFAIK it parses the command line and envvars and invokes the interpreter via python3X.dll.

And I think the /usr/bin/python symbolic link is apparently supposed to be managed using update-alternatives on Linux, which I guess is similar, but is not installed by default nor is it available on Mac.

This is actually a Debian (and naturally Ubuntu and anything else Debian-based) specific thing btw, other distributions don't have that.

Yes, symbolic links exist on Windows, but only for NTFS, not FAT32.

And IIRC there's a toggle setting to enable support for them :upside_down_face:

IMO, perhaps just following pip's lead and copying the executable stubs might be the lowest friction solution.

Well, pip's executable stubs (aka packaging entrypoint launchers) are a bit different, it's not as straight-forward here. Like I said above, I am pretty sure the executable is more than just a stub, so we need to create stubs.

...I guess asking for a default /usr/bin/python and /usr/bin/pip links to be added for Mac/*nix is probably out of scope for the issue, huh? ;)

No, if we are fixing the inconsistencies, I think it should be done for all platforms. On macOS it can be done via symlinks though, I am actually surprised you mentioned it doesn't already do this.

This is adding little to this issue. java/javac, dotnet, node/npm, gcc, clang, go, etc., etc. all are CLIs that are at least semantically identical cross-platform. There seems little reason for Python to be the outlier, especially as it is billed as an approachable introduction to coding.

I agree, it's unfortunate. If you are teaching introduction to Python, I would consider using IPython or Jupyter though. Anyway, I am glad to see you are advocating for your students, I wish my teachers did that when I was in college :sweat_smile:

[ptyork]

Thanks for the clarifications, @FFY00!

On macOS it can be done via symlinks though, I am actually surprised you mentioned it doesn't already do this.

Surprised me, too. I just installed the latest from python.org just to make sure:

% where python
python not found
% where python3
/Library/Frameworks/Python.framework/Versions/3.11/bin/python3
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3
/usr/local/bin/python3
/usr/bin/python3
% ls -l /usr/local/bin/python*
lrwxr-xr-x  1 root  wheel  70 Dec 16 14:48 /usr/local/bin/python3 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3
lrwxr-xr-x  1 root  wheel  77 Dec 16 14:48 /usr/local/bin/python3-config -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3-config
lrwxr-xr-x  1 root  wheel  78 Dec 16 14:48 /usr/local/bin/python3-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3-intel64
lrwxr-xr-x  1 root  wheel  73 Sep 23 12:19 /usr/local/bin/python3.10 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10
lrwxr-xr-x  1 root  wheel  80 Sep 23 12:19 /usr/local/bin/python3.10-config -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10-config
lrwxr-xr-x  1 root  wheel  81 Sep 23 12:19 /usr/local/bin/python3.10-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10-intel64
lrwxr-xr-x  1 root  wheel  73 Dec 16 14:48 /usr/local/bin/python3.11 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11
lrwxr-xr-x  1 root  wheel  80 Dec 16 14:48 /usr/local/bin/python3.11-config -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11-config
lrwxr-xr-x  1 root  wheel  81 Dec 16 14:48 /usr/local/bin/python3.11-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11-intel64
% ls -l /usr/bin/python*
-rwxr-xr-x  76 root  wheel  167120 Aug 24 04:59 /usr/bin/python3

Would also be nice to have it do the same for pip as there's only a /usr/local/bin/pip3 and pip3.x.

Debian/Ubuntu .deb package seems also not to add a /usr/bin/python or /usr/local/bin/python symbolic link. python3-pip DOES however have pip, pip3 and pip3.x script replicas. Different teams making the installers, I guess.

FWIW, I'll go to my grave not understanding the real distinction between /usr/bin and /usr/local/bin. 😆

If you are teaching introduction to Python, I would consider using IPython or Jupyter though.

Agreed!! We include Python in a (very) intro course, a "scripting and automation" course, database, and data science. Each kinda lends itself to different environments. Intro uses some kind of simple IDE (currently IDLE maybe?). Automation I think uses Mu, Data Science uses Jupyter. And my latest pain is in Database, where I need them to "experience" accessing a database from an application...here I use VSCode and CLI since they are connecting to a local MariaDB daemon.

I guess the challenge of having such a powerful, general purpose scripting language is that it is used in so many different ways...

[zooba]

I've been hesitant to add more executables to the Python install directory, because enough users already have messed up PATH that adding more is only going to be a bigger problem.

Any changes at all are going to cause churn, which is going to break people, which is going to use up any remaining goodwill we have left. I'd rather save the churn until we actually have a good, long-term fix. But I also don't know what that fix will be.

My best guess right now is that all the scenarios that would benefit from having "python3" be slightly more consistent (it's still not going to behave the same) would benefit far more from using a distro (who can easily do this themselves) or a preinstalled/configured environment such as via Jupyter or Minecraft. The python.org install of Python is such a long way from being a beginner friendly option that I'd rather leave that space clearly open for someone who wants to focus on that audience.

The Store install is a significant concession, but also properly handles the PATH modifications so I'm far less concerned about multiple executables (and the stub that launches the store was added by Microsoft in support - we don't control that from here, though I personally have the contacts to make changes if needed). That said, if they don't fix up the execution alias UI some time soon I'll seriously consider dropping some of the entry points.

[zooba]

enough users already have messed up PATH that adding more is only going to be a bigger problem.

To offer a really concrete example here:

• User installs (hypothetical) 3.11.2 with added python3.exe. 3.11 gets pushed to the front of PATH
• User installs security update to 3.10. 3.10 gets pushed to the front of PATH
• Now python.exe is 3.10 and python3.exe is 3.11. The only way to fix it is to manually modify PATH, and a user who knows how to do that could've set it up manually in a way that wouldn't break like this.

[FFY00]

I understand, and agree with your point regarding churn, however if we go that route I think we should decide what is our stance regarding installing versioned executable in general and document it, along with whatever exceptions we might have (eg. Windows if we decide that we want to install versioned executables as a general practice, or Linux if we decide we don't).

Having somewhere in the documentation where we can point people to learn what exactly is the difference in behavior between the different platforms would go a long way to mitigate this issue.

[ptyork]

• Now python.exe is 3.10 and python3.exe is 3.11.

and pip3.exe presumably would be 3.10. Yeah. That is a mess. Yuck!

As a "convenience" maybe coming up with a managed "standard" method of setting the current version that is consistent across platforms? Perhaps pysetver or something. It could present a list of known installs (in the path on Windows and common install locations elsewhere) and allow you to pick one to set as active? Would modify the path on Windows and update the symlink elsewhere? Kinda sorta like venv, but for the base install? Dunno. Might just be more churn...

Having somewhere in the documentation where we can point people to learn what exactly is the difference in behavior between the different platforms would go a long way to mitigate this issue.

Doesn't fix the copy/paste issue, but I agree that it is a good middle ground if no universal fix can be envisioned.

[zooba]

As a "convenience" maybe coming up with a managed "standard" method of setting the current version that is consistent across platforms? Perhaps pysetver or something.

This would be great, but unfortunately doesn't work well (on Windows at least). A process can't change the environment of its parent process, which would mean it would have to be a batch file and a PowerShell script and a Bash script to handle the three most popular shells (on Windows - Git Bash gets a lot of use), and nobody has signed up for that.

Would modify the path on Windows and update the symlink elsewhere

This is actually my ideal design, but it requires OS changes to work properly. I'm trying to argue those into existence before I give up and look for an alternative (potentially renaming the py.exe launcher to python.exe and python3.exe... which it already supports, but obviously the default counts for a lot... and I already know how this will break some users).

A non-terminal interface such as Jupyter, Mu or VS Code, is always going to avoid this level of confusion. It's a shame, but also pretty irreversible, that terminal skills are a prerequisite to using Python.

[ptyork]

A process can't change the environment of its parent process, which would mean it would have to be a batch file and a PowerShell script and a Bash script to handle the three most popular shells (on Windows - Git Bash gets a lot of use), and nobody has signed up for that.

True...but...to change the path permanently, you're modifying the registry using setx or similar. Which will require closing and reopening the command prompt anyway. So using C/Win32 or python + the winreg module could do this. You'd just need to print something like "Changes won't be effective until you close and reopen the command prompt" at the end.

Would be happy to throw together some proof-of-concept code for this if interested.

But, yes, a more elegant "work immediately" option would need to BOTH modify the registry and refresh the current environment from the registry. Sadly, an age-old issue since...ages old. Like me. :D And one that would indeed require a bat/ps1/sh script.

potentially renaming the py.exe launcher to python.exe and python3.exe

Interesting. Would be the "most similar" to symlinks on other platforms. But I can imagine this would almost have to be an "opt in" option. At least if other Python installs are detected.

No perfect solutions here, huh? At least without a Tardis.

[earonesty]

maybe python can ship with a "manage with pyenv" as the standard installation, so users who want some facility with managing versions and the whole python3, 3.X executable thing can have that managed for them, and we can decouple the "manage python version stuff" repo from "cpython". be nice if cpython wasn't even in the business of managing the active version, paths, etc.

[zooba]

I've proposed such a project at work, and there's a bit of interest in doing it. I do think the best path out of here is to let python.org Python keep doing its thing, but introduce something new for managing installs that works with the exact same runtimes, but not the original installers.

That way, people who keep downloading the same installer they always have been won't break, but we can start moving people who want a better experience onto the new thing, hopefully without forcing the burden of a totally new project onto the volunteer team.

[Timmmm]

• Now python.exe is 3.10 and python3.exe is 3.11. The only way to fix it is to manually modify PATH, and a user who knows how to do that could've set it up manually in a way that wouldn't break like this.

This is a valid concern but frankly I think it's less annoying than not having a cross platform way to run Python 3!!

E.g. in GitHub CI I can't use python or python3 because the former is Python 2 on Linux and the latter doesn't exist on Windows. That's clearly awful.

[chrisoldwood]

I came here after we ran into the same cross-platform problem as many others -- python3 is expected on Linux but doesn't exist in the (non-store) Windows installation. I can think of many solutions such as copy, mklink, shimgen, .bat, etc. but wondered what the pitfalls might be of each approach but haven't really found a nice succinct answer. (I assumed copying python.exe to python3.exe had been ruled out as it would have been done years ago if it really were that simple.)

This seems like it should be an issue long since reported, but I cannot find it by searching. So I apologize if this is a duplicate.

I was curious too 🙂 so had a good rummage through the issues as I know there are StackOverflow questions about this from years ago, because I've just read 'em. I found the following previous discussion from 2014 useful:

#65705: [Windows] installations should include pythonX.exe and pythonX.Y.exe executables

(originally titled " Windows MSI installer should mklink (symlink) python.exe to python2.7.exe" 😮.)

The earliest mention of this problem I could find here was in this comment in 2009: https://github.com/python/cpython/issues/51134#issuecomment-1093480223

Just when I think I'm close to understanding the trade-offs I discover #72872 py.exe ignored PATH when using python3 shebang and this comment:

Also worth noting is that the Microsoft Store package of Python does include versioned executables (because we're able to provide them without causing excessive clutter on PATH), but it does not include py.exe (because the versioning of that would get broken really quickly).

So now I'm really not sure what the best approach is even for running a script on Windows, let alone cross-platform wise 🤔.

[zooba]

So now I'm really not sure what the best approach is even for running a script on Windows

Unfortunately, the best way is for the person running it to know how they launch Python. This one also works cross-platform 😉

The next best way is for you to include a full copy of Python with your script and include a shortcut/launcher that makes it appear like a normal executable. (Pynsist and Briefcase both generate suitable installers, and some other tools like Pyoxidizer pack it into a single executable. The embeddable distro is meant for this purpose and is only ~8MB)

If you're in a situation where you have a trivial script (no installed dependencies) and you can't rely on your users to know how to launch Python, then yeah, this situation isn't great. But I'd contend that this is also the situation most likely to be harmed by inconsistent PATH, dependency conflicts, and such.

One day someone may contribute the support to concatenate a zip file to the py.exe launcher, which would then let you distribute an executable containing your script that assumes your users have already installed Python. I personally would never inflict that upon my users, but people seem to want to do it 🤷‍♂️

[ptyork]

The "reverse" solution to this is to add a common py launcher to the *nix + macos installers and just slowly migrate documentation towards using this instead of python3 or python. Maybe add a pyp as a standard pip launcher? I find this less desirable, but it doesn't break anything. The py launcher is venv-aware. And I've even found tutorials that oddly already use py in their examples...so there's a head start. ;)

This is basically how MS does it with dotnet. One common launcher with potentially many tens of versions of SDK and runtime.

Slightly changing the subject, the current windows store stub+install I find to be exceptionally confusing. Because it's the only Windows python3.exe, once installed it's the one that runs when you run python3 (duh). But running python will execute that version ONLY if you have no other version installed, since a standard python install will prepend the path before the ...\Microsoft\WindowsApps\ version

In other words, Installing a new Python3 version updates python but not python3, which would be SUPER confusing to a novice. And to add misery, both pip and pip3 now point to the new install, NOT the WindowsApps one. You'd need to run python3 -m pip (but NOT python -m pip) to manage packages for that version.

Plus an "aspiring to be savvy" user will get extra confused finding that, not only is the executable in an entirely different directory tree than the actual install, but the python3.exe executable is in the user directory while the binaries, packages, etc. are in the shared "program files" directory tree in a long and name-mangled directory (needing to run python3 and inspect sys.path to figure this out). I mean, it's a fun exercise, but logical and consistent? Not as much.

It's a bit of a mess if I'm being honest (and obviously I am). If there's a way to just wrap the regular installer into the MSIX package (I think there is), then that might be more desirable here. Anything to help remove inconsistency seems a worthy project.

[zooba]

Anything to help remove inconsistency seems a worthy project.

I agree, but I don't want to go breaking the world by changing stuff without being confident we're landing in a good spot. The regular installer has no shortage of issues, as does an MSIX install, as does using redirector executables, as does using symlinks, as does using PATH, as does having an annual incompatible release, etc.

Solving one problem risks making others worse (evidence: half the things I just listed were added to solve problems). And a lot of things are in motion that could fundamentally change everything again (e.g. proposed PEP 711). There's a growing consensus that globally installed runtimes are a bad design, which I've believed for a while but it's way more widely accepted now, especially among those who are the focal points for complaints about them.

While there are papercuts in the current situation, as long as it doesn't change people can figure out how to work within it. If we're changing it slightly every year, nobody will have a chance. So we're basically staying put until it's clearer what they way forward should be.

[Timmmm]

I don't think you will find a perfect solution for this. Your options are:

1. Fix it now and deal with the pain for a while.
2. Wait another 10 years and then fix it and deal with the pain.

Python installation is already enough of a total mess that adding python3.exe doesn't move it from "people understand this" to "it's confusing". It goes from "it's confusing" to "it's confusing but at least there's a cross-platform way to run Python3".

The best time to plant a tree...

[chrisoldwood]

Personally what I'm struggling to understand is why, if the Windows Store package includes a python3.exe [1], has the non-store package not also been changed to be consistent with it. i.e. how can fixing only the Windows Store package lead to less "chaos" than making both packages behave the same?

I freely admit I know very little about the Python ecosystem on Windows or Linux so appreciate that things are complicated but I don't believe you can ignore how people are invoking Python on Linux simply because this issue is about Python on Windows. Python is a cross-platform tool and teams are expecting fundamental stuff like python3 -V to work the same.

As an aside what brought me to this conversation was someone adding mypy as one of the linters to our (cross-platform) build pipeline. It's README exclusively mentions using python3 to invoke it.

---

[1] I'm basing that entirely on this comment in the Windows Store issue: https://github.com/python/cpython/issues/79158#issuecomment-1093802071 so maybe things changed after that and it got dropped. I fully understand why it's needed in the Windows Store package -- to replace the stub from Microsoft.

[zooba]

tl;dr: We assume people installing from python.org know how to launch Python.

how can fixing only the Windows Store package lead to less "chaos" than making both packages behave the same?

The main assumption is that people who choose to go to python.org to download an installer won't bother to get it from the Store. And a secondary assumption is that those who know to go to python.org also know how to launch Python (and so won't use python3, which has never worked in the past).

Clearly this second assumption is occasionally incorrect, as people who have installed from python.org still follow instructions that (apart from Microsoft's stub) won't ever have worked for them.

One other mitigating factor that makes it safer to add the extra executable to the Store install is that there can be only one python3.exe in that case - it doesn't modify PATH, it updates a special symlink - and so it's far less likely for a user to have python and python3 pointing to different versions (it's possible, but a user will have to have explicitly modified one and ignored the other, which appears right next to it in the same UI).

So assuming that most people who install the Store package have not installed from python.org, and are simply following instructions to get stuff to run, it makes sense and is safe to provide both python and python3 commands (and python3.x). The potential for conflicts or confusion between Python versions is significantly reduced, which is simply not possible with a PATH-based approach, and certainly not given the mixed history.

The best time to plant a tree...

I generally agree, but my experience of tree planting in Python installers so far has been a huge wave of literal hatred and demands for things to change back (or for me to lose my job, etc.). Please excuse me for being hesitant to volunteer to face that alone again.

[ptyork]

We assume people installing from python.org know how to launch Python.

Beating a dead horse, but I think this is probably not a 100% valid assumption. People get started with Python using a number of paths.

For example, my students are given the instructions for using WinGet, Homebrew, or downloading/installing. All install the Python.org version. And again, despite every effort to explain and differentiate instructions for Mac & Windows users, I will be hit with "why doesn't this command work?" questions multiple times on every assignment. Sure, maybe a few aren't destined to be superstar coders (or even very good at following instructions), but they shouldn't be frustrated and discouraged by this (see especially the pip issues bellow).

Other users might start a journey of learning Python on their own from one source. Stop. Restart after a while from another source. And both may give different "getting started" instructions.

I don't think a person installing from Python.org can be assumed to be experienced.

I'll also just clarify current Windows installer behavior for those not as familiar.

Not Installed (clean Windows install)

Both python3.exe and python.exe are located in %USERPROFILE%\AppData\Local\Microsoft\WindowsApps (I'll call this the user\WindowsApps dir). This is in (and stays in) the user's PATH by default. Running either of these will open the Windows Store and allow the user to install WindowsStore Python. Convenient.

WindowsStore Python Install

python3.exe and python.exe in user\WindowsApps dir are still there post install, but now link to the "real" Python install in c:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1008.0_x64__qbz5n2kfra8p0\ (your directory name will differ ... I'll just call it the pf\Python dir for short).

pip.exe, pip3.exe, pip3.11exe, python3.11.exe, pythonw.exe, pythonw3.exe and pythonw3.11.exe are also added to user\WindowsApps and linked to executables in pf\Python.

No changes are made to PATH, so the executables in user\WindowsApps are found if no other Python installs exist.

Note also that the py Python Launcher is not installed.

Python.org Install

This one installs python.exe and pythonw.exe in (e.g.,) %USERPROFILE%\AppData\Local\Programs\Python\Python310 (user\Python). pip.exe, pip3.exe, and pip3.10.exe are installed in a Scripts sub-directory.

Both user\Python and user\Python\Scripts are added to PATH immediately above user\WindowsApps.

This DOES install the handy py launcher, placing py.exe and pyw.exe in C:\Windows (again in the PATH by default and usually at or near the top).

So now, python launches the newly installed 3.10. The (also newly installed) py launcher targets the previously installed WindowsStore 3.11, as does every other python executable. Yuck, but not the end of the world.

EXCEPT now both pip and pip3 and pip3.10 target the 3.10 install. And you need the fully qualified pip3.11 to hit the WindowsStore 3.11 install. Meaning python and pip might behave as expected, but python3 and pip3 target different installations.

And it's further exacerbated if the more common novice mistake happens and both Python.org and WindowsStore versions are identical. In that case only python3 -m pip will target the WindowsStore version. (FYI, in this situation, it appears the py launcher will prefer the Python.org install over the WindowsStore one)

---

The above it obviously just one example. But similar chaos happens any time the WindowsStore and Python.org installers are mixed and matched. Which for a novice is FAR from unlikely. It's happened quite a number of times with my students. Usually in the opposite direction, having installed from python.org but unwittingly installing the WindowsStore version by typing python3 and just pressing the big green button.

The multiple ways of launching python is confusing and troubling, but the issues that arise from pip and pip3 ambiguities might be the most impactful. Certainly the hardest for a novice (or even a professor) to diagnose.

I generally agree, but my experience of tree planting in Python installers so far has been a huge wave of literal hatred and demands for things to change back (or for me to lose my job, etc.). Please excuse me for being hesitant to volunteer to face that alone again.

I TRULY hate that for you!! The pressure to please everyone when it is impossible is...terrible. And I can definitely relate to having your a$$ handed to you simply for doing what you thought was right or best. And it must be an order of magnitude worse given the size of the Python install base and how many are impacted by your decisions.

I also hear you that fixing the inconsistencies might break a small minority of users' code. And that even though this minority are also the ones most capable of diagnosing and fixing the issues (or maybe reading a README before jumping on an upgrade), they are the vocal ones who'll be coming for you with tar and feathers in hand.

And I COMPLETELY respect the desire to take your time to get it right. I won't pretend to know the best/right answer. This thread has certainly shown why it's tricky. But I do hope the team/community will consider "planting a tree" or "ripping off the band-aid" at a point in the near future.

I don't know if the formal PEP process can cover this. But perhaps a public vetting of a solution might help relieve some of the personal pressures. And at least allow the tar and feathers to be distributed more broadly.

[Timmmm]

as people who have installed from python.org still follow instructions that (apart from Microsoft's stub) won't ever have worked for them.

I'd just like to clarify that this isn't a case of "newbies following Linux instructions that obviously won't work on Windows". python3 should work on Windows. I don't think you contradicted this explicitly I just want to make extra clear that the motivation for this is not primarily to help newbies (though obviously that is a big benefit).

One other mitigating factor that makes it safer to add the extra executable to the Store install is that there can be only one python3.exe in that case - it doesn't modify PATH, it updates a special symlink

Interesting, but doesn't that mean you can get the python/python3 version mismatch issue now (if users install Python from the store and website) and that adding Python3.exe to the website installer will make it less confusing already in that case?

[zooba]

We assume people installing from python.org know how to launch Python.

Beating a dead horse, but I think this is probably not a 100% valid assumption. People get started with Python using a number of paths.

Granted, but what alternative do we have? Assume that people using the installer don't know how to launch Python? What then? Add a message to the installer (oh look, it's already there)? Add links to getting started docs (also already there)? Restrict the ways Python can be launched to avoid conflicts (but all the ways are currently used by someone)?

The vast majority of users of the installer do know how to launch Python, because they already have been. Those who are completely new must have used some kind of reference to find the site, and so whoever pointed them that way can also mention it (or they can look at the site or the installer, which recommend py fairly prominently).

The multiple ways of launching python is confusing and troubling, but the issues that arise from pip and pip3 ambiguities might be the most impactful. Certainly the hardest for a novice (or even a professor) to diagnose.

The same issue arises frequently on Linux, and so the general recommendation for a few years now has been to use python -m pip unless you're in a virtual environment (i.e. you know you've configured your terminal PATH already), where python is whichever command is needed for the Python that you want to run.

I don't know if the formal PEP process can cover this. But perhaps a public vetting of a solution might help relieve some of the personal pressures.

That's the theory behind the distutils removal. We'll see how much fallout comes my way when 3.12 ships (we all agreed it needed to be done, but my name's on the PEP, even though others did the actual removal).

Interesting, but doesn't that mean you can get the python/python3 version mismatch issue now (if users install Python from the store and website) and that adding Python3.exe to the website installer will make it less confusing already in that case?

There are a million other ways to get into this situation though, and the only good way out of it is for a user to learn how to launch the Python they want, or to learn to install a different tool and launch a different command. Adding python3.exe to the regular installer makes this situation worse for the next 5+ years (3.12 and 3.13 installed side-by-side is far more likely than 3.13 Store and python.org), and potentially leaves it worse after that as well.

I acknowledge that it may be better after that transition period. But given the sheer amount of discussion going on right now about changes like these and on similar timescales, I'm not going to make any changes right now without that side of things settling.

[Meinersbur]

To offer a really concrete example here:

• User installs (hypothetical) 3.11.2 with added python3.exe. 3.11 gets pushed to the front of PATH
• User installs security update to 3.10. 3.10 gets pushed to the front of PATH
• Now python.exe is 3.10 and python3.exe is 3.11. The only way to fix it is to manually modify PATH, and a user who knows how to do that could've set it up manually in a way that wouldn't break like this.

Counterpoint: I have the MSYS2 bin path in my %PATH%[^1] and MSYS adds Windows directories into ${PATH}[^2]. When installing python in MSYS (pacman -S python) it installs a MinGW-compiled python3.exe into that directory, as expected by UNIX environments. In the Windows command line, python will now call the Windows version (right now 3.11.4 for me) and python3 calls the MSYS2 version (3.11.3).

In contrast, your example is only ephemeral until those old versions go out of support. Also, I would consider the issue that a security update modifies %PATH% causing the system to break because it downgrades the default version from 3.11 to 3.10 (e.g. programs using features introduced in 3.11). Not just python3.exe, but also any executable in /Scripts.

[^1]: To have GNU tools available on the Windows command line. Required e.g. when building LLVM. Alternatives are Gnuwin32 or GitForWindows. The former is very old, the latter itself based on MSYS2. [^2]: With the MSYS2_PATH_TYPE=inherit option.

[chrisoldwood]

User installs security update to 3.10. 3.10 gets pushed to the front of PATH

This strikes me as an odd thing to do. Why would the 3.10 installer move the position of itself in the PATH when updating? If it's already in the PATH it should leave it alone as the user may have re-ordered their paths on purpose. Is there a good reason an update (not a new installation) should do this? Or is this just an artefact of the way the installer works when updating, e.g. remove entirely, then add like new?

[tony-p]

A non newbie/tutorial use case is that in conda python is packaged very minimally meaning even in conda there is no python3 alias (eventough it is a virtual env).

There are then other packages packaged with scripts that use a python3 shebang as they were primarily developed on Linux and it just works. However when called from an env on Windows it either doesn't work or calls the global install. Then the only real option (without manually modifying the environment) is to wrap the call with python to the qualified path of the script within the env install rather just being able to call the cmdlet from within the environment.

However I spotted above, that shebangs potentially bypass the path entirely on windows which also seems like a big problem.

[zooba]

Giving this some more thought, I think we can probably make venv create a python3.exe command, and at least make this work in an active venv. @vsajip - any thoughts? We'd probably need a --without-python3 option for compatibility, but really I'd be surprised if anyone is relying on python3 not working inside a venv.

Because a venv isn't active by default, and when you activate one it puts itself at the front of PATH, this misses my concerns about having the installer modify PATH. And it also overrides a Store install (or integrates with one if you create the venv from a Store install), which is otherwise potentially going to cause confusion when it "works" but launches the wrong thing.

[zooba]

I also just posted #114881 which I think could help cover some of the gap for users who want safer PATH settings. Interested in people's thoughts over there on whether it would be usable enough (one day it could become the default, but it's one of these scenarios where we need to inform people who made their code work 5 years ago and stopped listening to anything we say, so they won't find out until they - or more likely, their users - break...)

[Timmmm]

I still think your concerns about backwards compatibility are unwarranted. The current behaviour is a total mess. What's the point of being backwards compatible with that?

IF there was no risk of mixing up Python versions currently, and adding python3.exe introduced that risk, then yeah fine. But that's not the case. Pythons can already get mixed up today.

Here are your options:

Option	Python PATH behaviour today	Python PATH behaviour for the next 5 years	Python PATH behaviour after 5 years
Do nothing	Broken	Broken	Broken
Add python3.exe to venv only	Broken	Broken	Broken
Add python3.exe	Broken	Broken	Fixed!

I mean, fixing it in venv is probably better than nothing (though it does add a surprising behaviour which is never good). But ... just add python3.exe and accept the fact that it won't make PATH issues go away until the old versions are dead.

[zooba]

The problem with that table is that Python PATH behaviour will still be broken after 5 years. The way to fix it is to stop modifying PATH through the normal installer and only do it via the launcher (which adds exactly one directory to PATH and hopefully can make it so all Python installs share it - see #114881).

Adding python3.exe now just makes all of that breakage worse, adds new surprising behaviour (moreso than adding it to venvs, which you have to explicitly activate), and doesn't actually lead to it being fixed.

Adding python3.exe to the launcher (an alias of py.exe -3) would be a slightly better solution than adding it to the main install directory. It's still going to break people, but at least the launcher is optional.

[jaraco]

Pardon the drive-by comment, but IMO the python3 name is an aberration, an artifact of a transition that's since passed. At the language summit, it was floated to retire the name, but at least one dev (Guido) advocated for keeping it because of the brand recognition. Let's work to reduce the variance on platforms by converging on a simplified experience (use python or even better py) instead of converging on a proliferation of names (python, python3, python3.14, ...).

[Timmmm]

The way to fix it is to stop modifying PATH through the normal installer and only do it via the launcher

Well no, because the launcher is only available on Windows. That's the whole point of this issue. I suppose if you can convince all the Mac and Linux distros (Homebrew, Ports, Debian, Fedora, Arch, etc. etc.) to add a py launcher that might be a good solution. That will take much longer than just fixing it on Windows though.

Pardon the drive-by comment, but IMO the python3 name is an aberration, an artifact of a transition that's since passed.

Absolutely, but this problem is already awkward enough. There's no way it's worth adding additional difficulty by trying to use a "nice" name.

[jaraco]

Well no, because the launcher is only available on Windows. That's the whole point of this issue. I suppose if you can convince all the Mac and Linux distros (Homebrew, Ports, Debian, Fedora, Arch, etc. etc.) to add a py launcher that might be a good solution. That will take much longer than just fixing it on Windows though.

I think Brett's work on python launcher gets pretty close. It's already part of homebrew, linuxbrew, Fedora, and Arch, leaving only Ports and Debian from your list, but it's also trivially installable on Linux distros. Moreover, the project has ambitions to support Windows at which point it could be a drop in for the Python Launcher for Windows and possibly distributed with CPython. Maybe our energy would best be spent focusing on delivering that more easily to more platforms and completing the Windows support.

[Timmmm]

A non-goal of this project is to become the way to launch the Python interpreter all the time.

(Though I don't really see why it couldn't tbh.) If it could be distributed with CPython then that would be great, but that's a whole lot of ifs and work that isn't even planned... If it were me I would just add python3.exe now, as Microsoft has already done, and then try and do a long term fix.

But I've banged on about this enough now; I think you get the message! Let's reconvene in 5 years 😄

[tony-p]

I think Brett's work on python launcher gets pretty close. It's already part of homebrew, linuxbrew, Fedora, and Arch, leaving only Ports and Debian from your list, but it's also trivially installable on Linux distros. Moreover, the project has ambitions to support Windows at which point it could be a drop in for the Python Launcher for Windows and possibly distributed with CPython. Maybe our energy would best be spent focusing on delivering that more easily to more platforms and completing the Windows support.

How does this work with virtual environments? From what I understand, the current windows launcher will also override python versions from a virtual environment when not directly called (shebangs, via tooling etc). OS wide installs are the least controlled way to use python (or many other tools) and therefore virtual environments are used more and more to create this certainty. Pushing solutions that continue to work around the problem and undermine virtual environments don’t really help.

[jaraco]

How does this work with virtual environments?

See the searching for interpreters guidance for details, but tl;dr, it has virtualenv support.

[zooba]

at which point it could be a drop in for the Python Launcher for Windows and possibly distributed with CPython

Afraid not, Brett's launcher has about 1% of the functionality of our current one, and he has no desire to add the rest (nor would I recommend adding the rest - shebang emulation turns out to be nearly impossible to get right). It could certainly be distributed on other platforms though.