search

Homebrew / brew

🍺 The missing package manager for macOS (or Linux)
https://brew.sh
BSD 2-Clause "Simplified" License
40.13k stars 9.41k forks source link

pyenv does not work with `brew pyenv-sync` #17563

Open PheelaV opened 1 week ago

PheelaV commented 1 week ago

brew doctor output

Your system is ready to brew.

Verification

brew config output

HOMEBREW_VERSION: 4.3.7
ORIGIN: https://github.com/Homebrew/brew
HEAD: 43eaeca50fe3b6c755f3fc9bc6f43669e0db1039
Last commit: 2 days ago
Core tap JSON: 25 Jun 23:34 UTC
Core cask tap JSON: 25 Jun 23:34 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 12
Homebrew Ruby: 3.3.3 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.3/bin/ruby
CPU: dodeca-core 64-bit arm_lobos
Clang: 15.0.0 build 1500
Git: 2.39.3 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 8.6.0 => /usr/bin/curl
macOS: 14.5-arm64
CLT: 15.3.0.0.1.1708646388
Xcode: N/A
Rosetta 2: false

What were you trying to do (and why)?

brew pyenv-sync

I am trying to use brew-installed python distributions using pyenv.

What happened (include all command output)?

but when I execute python or pyenv which python I get

➜  .pyenv python
zsh: command not found: python
➜  .pyenv pyenv which python
pyenv: python: command not found

I see the symlinks:

ls ~/.pyenv/versions
3.10.0  3.10.11 3.10.14 3.10.4  3.10.7  3.11.0  3.11.3  3.11.6  3.11.9  3.12.2
3.10.1  3.10.12 3.10.2  3.10.5  3.10.8  3.11.1  3.11.4  3.11.7  3.12.0  3.12.3

Then I see

pyenv versions
  system
  3.10.0 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.1 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.2 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.3 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.4 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.5 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.6 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.7 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.8 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.9 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.10 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.11 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.12 --> /opt/homebrew/Cellar/python@3.10/3.10.14
  3.10.13 --> /opt/homebrew/Cellar/python@3.10/3.10.14
* 3.10.14 --> /opt/homebrew/Cellar/python@3.10/3.10.14 (set by /Users/filip/.pyenv/version)
  3.11.0 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.1 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.2 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.3 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.4 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.5 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.6 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.7 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.8 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.11.9 --> /opt/homebrew/Cellar/python@3.11/3.11.9
  3.12.0 --> /opt/homebrew/Cellar/python@3.12/3.12.4
  3.12.1 --> /opt/homebrew/Cellar/python@3.12/3.12.4
  3.12.2 --> /opt/homebrew/Cellar/python@3.12/3.12.4
  3.12.3 --> /opt/homebrew/Cellar/python@3.12/3.12.4
  3.12.4 --> /opt/homebrew/Cellar/python@3.12/3.12.4

This is happening both when using a .python-version file via pyenv local 3.10 as well as for the global option pyenv global 3.10

As you can see, the correct python version is selected as denoted by the asterisk.

What did you expect to happen?

I expected that pyenv would select and appropriate pyhton version (brew-installed) and carry on.

This is relevant? /orgs/Homebrew/discussions/4664

Step-by-step reproduction instructions (by running brew commands)

1) brew install python@3.10

2) brew install pyenv

3) pyenv --init makes me put this in path, I have updated it with the shims and put it into my ~/.zshrc

>>> pyenv init >>>

export PYENV_ROOT="$HOME/.pyenv" [[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH" [[ -d $PYENV_ROOT/shims ]] && export PATH="$PYENV_ROOT/shims:$PATH" eval "$(pyenv init -)"

<<< pyenv init <<<<

4) source ~/.zshrc

5) pyenv which python; python --version

rrotter commented 1 week ago

I'm not sure how this ever worked unless the layout of the python formulae or the behavior of pyenv has changed since the pyenv-sync command was added. No python formula links python in the bin dir, only the version with the version number appended (python3, python3.12, etc), and only the newest version links python3, the rest only link python3.xx:

~ % ls $(brew --prefix)/Cellar/python*/*/bin/python* | grep -v config
/opt/homebrew/Cellar/python@3.10/3.10.14/bin/python3.10
/opt/homebrew/Cellar/python@3.11/3.11.9/bin/python3.11
/opt/homebrew/Cellar/python@3.12/3.12.4/bin/python3
/opt/homebrew/Cellar/python@3.12/3.12.4/bin/python3.12
/opt/homebrew/Cellar/python@3.9/3.9.19/bin/python3.9

The other links are present, but only under libexec/bin, which I assume pyenv doesn't read. If this is to be fixed I think it's either asking pyenv to use libexec/bin (probably not realistic) or building an additional tree of symlinks with a complete bin dir (presumably in the python formulae, but possibly in the pyenv-sync command) for pyenv-sync to link to. (I suppose a third option is installing a hacked version of pyenv, but that seems like the most problematic solution)

DConcord commented 1 week ago

I'm seeing the same behavior (no "python" command available, only "python3"). But I'm also seeing the that "pyenv local", "pyenv shell", and even pipenv doesn't work with any version of python linked/synced from brew; only versions installed by pyenv

rrotter commented 1 week ago

Yep. pyenv only loads what's in .pyenv/versions/$PYENV_VERSION/bin/. In the case of the two latest homebrew pythons that is:

~ % ls .pyenv/versions/3.12.4/bin
2to3            pydoc3.12
2to3-3.12       python3
idle3           python3-config
idle3.12        python3.12
pip3            python3.12-config
pip3.12         wheel3
pydoc3          wheel3.12
~ % ls .pyenv/versions/3.11.9/bin/
2to3-3.11       python3.11
idle3.11        python3.11-config
pip3.11         wheel3.11
pydoc3.11

Note that there is no overlap in these lists (i.e. no python3 under 3.11.9). That's a feature of the python formulae so they don't conflict with one another, but it's also why they don't work with pyenv.

reitermarkus commented 6 days ago

That's a feature of the python formulae so they don't conflict with one another, but it's also why they don't work with pyenv.

Exactly that. Note that Homebrew is not a version manager. The only reason versioned formulae exist is because other formulae don't work with the newest version.

PheelaV commented 6 days ago

Indeed, something may have changed. I have noticed ~/.pyenv/bin does not get created (unless $pyenv install). As I was not exactly sure how pyenv is handling things behind the scenes, I decided to open this issue.

For the time being I have just resolved to use pyenv with redundancy.

Inspecting again, I can see 

~ % ls ~/.pyenv/versions
3.10.14 3.11.9  3.12.4
~ % ls ~/.pyenv/versions/3.12.4
bin     include lib     share
~ % ls ~/.pyenv/versions/3.12.4/bin
2to3              idle3.12          pydoc             python-config     python3.12-config
2to3-3.12         pip               pydoc3            python3           python3.12-gdb.py
idle              pip3              pydoc3.12         python3-config
idle3             pip3.12           python            python3.12

@rrotter Now I understand your comment

I'm not sure how this ever worked unless...

~ % /opt/homebrew/Cellar/python@3.12/3.12.4
Frameworks            LICENSE               bin                   sbom.spdx.json
IDLE 3.app            Python Launcher 3.app lib                   share
INSTALL_RECEIPT.json  README.rst            libexec
~ % ls /opt/homebrew/Cellar/python@3.12/3.12.4/bin
2to3              idle3.12          pydoc3            python3-config    wheel3
2to3-3.12         pip3              pydoc3.12         python3.12        wheel3.12
idle3             pip3.12           python3           python3.12-config

I would like to start playing with authoring/modifying homebrew formulaes, but do not have the time now (until September). In python@3.12 formulae Install unversioned symlinks in libexec/bin. Perhaps the best solution as you say is to re-create the pyenv installation structure using the contents /libexec/bin.

Possibly off topic, but providing context: My own usecase is to use brew-installed python versions to seed my poetry-based projects for development, which does require for the base version installation to exist on a system to create its venv.

I have seen people both

  1. use brew-installed python versions with pyenv (via manual/script linking)
  2. use pyenv-installed python versions with homebrew

I was leaning towards 1. as I want ideally one place to think about installing/uninstalling software, but I was seeking automation and brew pyenv-sync was promising.

@reitermarkus I see your point, if you are fine with latest minor releases use homebrew, if you need something more specific go elsewhere (e.g. pyenv/virtualenv/conda/docker).

Relevant #2138 Original PR 15507

MikeMcQuaid commented 6 days ago

Perhaps the best solution as you say is to re-create the pyenv installation structure using the contents /libexec/bin.

This seems like a good idea and the best implementation here.