Mac installation: since changes in Homebrew for Python 3.12, `pip3 install manim` doesn't work. Improvement of the doc is needed.

quark67 commented 3 months ago

Preliminaries

[x] I have followed the latest version of the installation instructions.
[x] I have checked the installation FAQ and my problem is either not mentioned there, or the solution given there does not help.

Description of error

The command pip3 install manim throw this error in the Terminal on Mac (Python is installed by Homebrew, as suggested in the Manim manual: https://docs.manim.community/en/stable/installation/macos.html):

error: externally-managed-environment

× This environment is externally managed ╰─> To install Python packages system-wide, try brew install xyz, where xyz is the package you are trying to install.

If you wish to install a non-brew-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip.

If you wish to install a non-brew packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification.

This is related to PEP 668 (https://peps.python.org/pep-0668/).

This is documented in this discussion on the GitHub for the Homebrew package manager project: https://github.com/orgs/Homebrew/discussions/3404

On February 21, this change was implemented, as writed here:

@pradyunsg we have now implemented this starting with Python 3.12: https://github.com/Homebrew/homebrew-core/pull/150390, and Python 3.12 has become the default Python 3 in Homebrew so more users will be made aware of the change.

Installation logs

Terminal output

``` error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try brew install xyz, where xyz is the package you are trying to install. If you wish to install a non-brew-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. If you wish to install a non-brew packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. ```

System specifications

System Details

- OS (with version, e.g., Windows 10 v2004 or macOS 10.15 (Catalina)): - RAM: - Python version (`python/py/python3 --version`): - Installed modules (provide output from `pip list`): ``` macOS 14.3.1 (Sonoma) 64 GB Python 3.12.2 Brotli 1.1.0 certifi 2024.2.2 gpg 1.23.2 mutagen 1.47.0 openvino 2023.3.0 pip 24.0 pycairo 1.26.0 TBB 0.2 wheel 0.42.0 ```

LaTeX details

+ LaTeX distribution (e.g. TeX Live 2020): TeX Live 2024 (MacTeX 2024) + Installed LaTeX packages: a list of 4693 packages

FFMPEG

Additional comments

I suggest an improvement of the manual for the Mac installation, specifically for users which have Python 3.12 installed. By the way: I have Python 3.11 installed and it's the command brew install py3cairo ffmpeg (see https://docs.manim.community/en/stable/installation/macos.html#required-dependencies) which has upgraded the Python version. During the installation of py3cairo and ffmpeg, I read this:

==> Installing py3cairo dependency: python@3.12
==> Downloading https://ghcr.io/v2/homebrew/core/python/3.12/manifests/3.12.2_1

The documentation can be improved either by replacing pip3 install manim with ~~brew install manim~~ (not working, unlike to what the text error suggest in the Terminal after the command pip3 install manim is entered), or pip3 install manim --break-system-packages (not tested yet, I will wait for your suggestion) or configuring pip.conf with break-system-packages by default (I don't know how to do this, I have never modified this file ; this is suggested here).

What do you suggest to do yet, before the manual is updated? How can I reinstall Manim (it was installed with a previous version of Python, but in every update of Python, I loose Manim, I have the ModuleNotFoundError: No module named 'manim' error)?

behackl commented 3 months ago

Ah, yes, this is a problem.

The proper solution and what homebrew expects you to do now is to use a virtual environment in which manim can then be installed. That is, if you have a directory for your manim projects use the terminal to navigate there, then run

python3 -m venv .py3env --prompt=manim-env

where .py3env specifies the (local) directory in which the Python venv will be installed, and manim-env is displayed in your terminal whenever the environment is activated. Both values can be chosen freely, the prompt can also be omitted (it will then just use the directory name).

Then, to activate the environment you type

source .py3env/bin/activate

in your terminal. From there on, the terminal should display the chosen prompt in the beginning of each new line.

Now you can run

pip install manim

without running into the "externally managed" problem.

quark67 commented 3 months ago

Thanks. I don't want to use virtual environment (too complex, more things to learn). This change from PEP 668 is for power users.

I have tried this: Creating a file pip.conf at ~/Library/Application Support/pip (needed to create the folder pip which don't exist), as explained here: https://pip.pypa.io/en/stable/topics/configuration/ (for the Mac).

Content of the file:

[global]
break-system-packages = true

Then, run pip3 install manim.

And it works! Caveat: an error message about openvino (??? why? According to https://en.wikipedia.org/wiki/OpenVINO it's related to IA).

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. openvino 2023.3.0 requires openvino-telemetry>=2023.2.1, which is not installed.

The installation message ended with this successfuly message:

Successfully installed Cython-3.0.9 Pillow-9.5.0 Pygments-2.17.2 charset-normalizer-3.3.2 click-8.1.7 click-default-group-1.2.4 cloup-2.1.2 decorator-5.1.1 glcontext-2.5.0 idna-3.6 isosurfaces-0.1.0 manim-0.18.0 manimpango-0.5.0 mapbox-earcut-1.0.1 markdown-it-py-3.0.0 mdurl-0.1.2 moderngl-5.10.0 moderngl-window-2.4.4 multipledispatch-1.0.0 networkx-3.2.1 numpy-1.26.4 pydub-0.25.1 pyglet-2.0.14 pyobjc-core-10.2 pyobjc-framework-Cocoa-10.2 pyrr-0.10.3 requests-2.31.0 rich-13.7.1 scipy-1.12.0 screeninfo-0.8.1 skia-pathops-0.8.0.post1 srt-3.5.3 svgelements-1.9.6 tqdm-4.66.2 typing-extensions-4.10.0 urllib3-2.2.1 watchdog-3.0.0

But when I test a simple file, I obtain this error:

Traceback (most recent call last):
  File "/usr/local/bin/manim", line 5, in <module>
    from manim.__main__ import main
  File "/usr/local/lib/python3.12/site-packages/manim/__init__.py", line 6, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

I see at https://github.com/ManimCommunity/manim/issues/3585 that I'm not the only one.

After running pip3 install setuptools (as suggested in issue #3585), all works fine (it seems).

Can I suggest that you add also a tip in the manual about the missing pkg_resources that is solved with pip3 install setuptools? According to https://github.com/ManimCommunity/manim/issues/3585#issuecomment-1901183355, pkg_resources is not automatically pre-installed in some versions of Python.

Thanks for your valuable help.

behackl commented 3 months ago

I don't want to use virtual environment

Then you will have to do exactly what you did, running --break-system-packages (which sounds worse than it actually is). But otherwise, with PEP 668 being implemented for Homebrew there is no other way.

Pyenv might be a reasonable alternative at this point too.

Can I suggest that you add also a tip in the manual about the missing pkg_resources

No need, we've removed the import of pkg_resources, this change will not be necessary with the next release.

Toby-Shi-cloud commented 3 months ago

Can we publish manim as a brew-packaged application? I am not sure if it is good

behackl commented 2 months ago

Can we publish manim as a brew-packaged application? I am not sure if it is good

I've wanted to investigate how other small-ish Python libraries handle this. Do you happen to know any examples? There aren't so many of us developers currently active, I'd rather avoid creating yet another responsibility for us to maintain perpetually.

Toby-Shi-cloud commented 2 months ago

I've wanted to investigate how other small-ish Python libraries handle this. Do you happen to know any examples? There aren't so many of us developers currently active, I'd rather avoid creating yet another responsibility for us to maintain perpetually.

I am not familiar with this. However, I found this post has tons of solutions for this question. Maybe we can just update the docs.

ManimCommunity / manim