Near impossible to get started with Python on macOS

[joakim]

Is your feature request related to a problem? Please describe. I received an email from my brother who's not a developer.

There is something wrong with developers.

I found a great Python script on a forum, where they said you just have to install Python 3.7+ and Pillow (pip install pillow) and voila.

If only it was that simple! I spent four hours trying to get it to work – installing, searching, reading question after question on Stack Overflow, without finding a good answer to something as simple as: How to get a Python 3 script to run on macOS.

Even after having installed Python 3 with the official Mac installer, macOS insists on using Python 2 (which was officially deprecated a few days ago). The answers point to all different directions, with everyone being so assertive and nobody able to agree.

In the end I found a blog post by a Red Hat guy who had struggled with the same problem. Even he thought he had found the best solution, until a Python guru suddenly appears out of the blue with what actually is the best solution, one that apparently only he and other Python gurus knows about. And when I tried that, it worked at once!

I find it incredible that in 2020, you can download an official Python installer for macOS, which won't work, won't inform you that it won't work, and won't say what you should do to make it work.

Describe the solution you'd like Don't expect people to magically understand all this. At least inform on the website what additional steps are needed before you can start using Python 3 on macOS. Ideally, fix the installer for macOS – include pyenv or something equivalent.

Describe alternatives you've considered Stop recommending Python to new developers.

Additional context I'm a developer myself and would've known to look for a version management tool, something I'm familiar with from Ruby and Node. We developers take knowledge like that for granted, but everyone's not like us. It's easy to forget that we're the weird ones, living in a bubble. It should be easy for others to join that bubble, not a struggle.

[Mariatta]

Thanks for the report with this issue.

pillow is a third party package. More information about installing third party packages (like pillow) using pip is described in https://packaging.python.org/tutorials/installing-packages/

The tutorial is quite lengthy, and I can understand if newcomers skipped reading that. Thankfully, there is now a "quick and dirty guide on how to install packages using pip": https://snarky.ca/a-quick-and-dirty-guide-on-how-to-install-packages-for-python/

You suggested: inform on the website what additional steps are needed before you can start using Python 3 on macOS.

You can start using Python 3 on MacOS after the using installer! I think the real problems are surrounding the third party package:

• newbies don't know how to install a third party package like pillow using pip

• we're telling newbies to "just pip install it", when in reality it is never "just" pip install, but rather, they need to first "create a virtual environment", and then "activate it". (also, they should be doing python -m pip install instead of just pip install, reason here)

Note that we already provide link to Python Packaging index in the installer, and in our download page (see screenshots), and the tutorial on how to install is linked from there.

So now we have another problem:

• newbies don't realize they need to go to Packaging Index in order to learn more about third party packages and how to install

What can we do to improve this experience?

• Since we already mention the "Python Package Index" in the download page ( at the very bottom), will it help if we move that higher up, maybe before the PGP keys section?

• Would it help if we provide link to the tutorial and quick start guide from the download page? This way, newbies have easy access to those learning resources instead of having to first go to PyPI.org, or having to search for it on the internet.

• Similarly, should we link to the tutorial from the installer?

• Is the download page or the installer the right place to teach people about pip and third party packages? PyPA already own and maintain the tutorial, and we also have some writeup about installing Python module from docs.python.org.

[joakim]

The problem wasn't that he couldn't pip install pillow, but that the Python script just wouldn't run, even though everything was installed as per instructions.

The issue is that a virtual environment is needed, as you mention. That's what has to be addressed. Learning how to install packages with pip doesn't help much if the script won't run. Telling people to run python -m pip instead of just pip is not very user friendly either.

XKCD sums up this issue nicely. That's one giant hurdle to overcome for someone new to Python. Shouldn't this be solved by the installer, ideally?

[onlinejudge95]

@joakim While downloading python interpreter, only thing that python.org is guaranteeing is well a python interpreter. Now if you look closely the concept of a virtualenv is more of a better software development practice rather than something tied to the python interpreter. Regarding

that the Python script just wouldn't run, even though everything was installed as per instructions.

Did you try any debugging from your end, if not please provide certain information(logs, traceback etc) for others to help you out.

[berkerpeksag]

The root cause of the problem is not specific to Python. You can have similar issues with other programming languages if the platform you are using provides system version of your choice of programming language.

There are several solutions to tackle this problem in the Python ecosystem. pyenv is one of them like you mentioned. Virtual environments are an option as well. There are tools like pipx that handles creating virtual environment part for you. You can also install third-party packages via pip install --user or python -m pip install --user.

And some Linux distributions try to solve this on their side. I remember there was some proposals to have separate system-x executables. I haven't followed all news so my knowledge on this subject might be outdated.

Unfortunately, it's not an easy problem to come up with one solution to fix it for everyone due to number of different programming languages, packages systems, distributions and operating systems.

[joakim]

Did you try any debugging from your end, if not please provide certain information(logs, traceback etc) for others to help you out.

The issue was immediately solved with pyenv. The "bug" is that finding this solution was so hard.

It's true that this may also apply to other programming languages. But I have the impression that it is more common with Python than for example with Node (JavaScript) due to backward compatibility issues and being shipped with macOS in this case.

If a version management / virtual environment tool is needed to run a large percentage of scripts that a user is likely to encounter or write, then I think it is only fair to inform the user of that issue and how to solve it. If it can't be solved by the installer itself, then informing users as they go to download Python is better than nothing. This website does a decent job of explaining the issue and linking to solutions.

This issue is really about barrier to entry. You want as few hurdles as possible during the first moments of interacting with Python, or the user will leave with a horrible first impression of the language. It doesn't matter if it's the best programming language in the world if you can't use it.

[onlinejudge95]

@joakim true that, your problem is understandable as a python user myself. But then i guess if it's my first time interacting with Python, then the complexities of running a script requiring a separate virtual environment would not be the first problem i would tackle if i am a newbie or else i would know what question to ask google.

See if we think of programming as an abstraction and languages just as implementation difference then don't you think anything non-specific to an implementation can easily be fetched provided we have right space to search it in?

[joakim]

@onlinejudge95 In principle, yes. In practice, I think it's in Python's own interest to make that information available to users either before or as they encounter that roadblock.

[ned-deily]

Thanks for the report. I think the main issue here is the continuing confusion between Python 2 and Python 3 command names and scripts, something not unique to macOS installations of Python. The confusion here could have been avoided by better documentation of the differences between using the command nanespython, python2, and python3 and similarly pip, pip2, and pip3. Yes, use of virtual environments is one way to avoid the differences, in that an active Python 3 virtual environment provides a python and pip alias to the Python 3 versions. But use of a virtual environment isn't necessary in situations like this; all that is required is ensuring that you are using the instance of pip that is associated with the instance of Python in use. If one is invoking python with python3 then usually there will be a corresponding pip3 (likewise, for example, python3.8 and pip3.8):

pip3 install pillow

But, even better, you can avoid any ambiguity by invoking pip via the python command line:

python3 -m pip install pillow

Unfortunately, our documentation in this area lags somewhat. With the retirement of Python 2, there are some on-going efforts to improve the docs but there is still much to be done.

[joakim]

I believe you are right, thanks for the clear explanation!

The tutorial page Using the Python Interpreter just barely touches on this topic in a footnote:

On Unix, the Python 3.x interpreter is by default not installed with the executable named python, so that it does not conflict with a simultaneously installed Python 2.x executable.

Here's a revised suggestion: Have a paragraph on that page dedicated to this topic (ie. how to correctly use python and pip from the command line with multiple versions installed). Link to this tutorial page from the Getting Started page, the wiki and the Python installer itself.

Basically, treat this topic as something that is fundamental to grasp in order to get started with Python.

[mwichmann]

This isn't going to help anyone except to provide a bit of comic relief... apparently getting it working on Mac is a challenege everywhere: https://xkcd.com/1987/

[hoefling]

Just my 2c: I'm not sure why lots of people praise pyenv so much, in my experience switches between incompatible ABIs result in errors a new user will surely not be able to solve (example: yet another question on SO recently).