search

conda-forge / python-feedstock

A conda-smithy repository for python.
BSD 3-Clause "New" or "Revised" License
46 stars 105 forks source link

conda-forge moving from Python 3.13 to 3.13t #738

Open joseavegaa opened 3 weeks ago

joseavegaa commented 3 weeks ago

Solution to issue cannot be found in the documentation.

Issue

Issue Description

When trying to install packages or use update --all in a conda environment with Python 3.13.0 installed, the installer tries to move the packages to the free threated version of Python 3.13, which is Python 3.13t. I currently do not know if it is only an issue with Windows, or if it affects other OS as well. Also, I am not sure if it's a one-way issue, or if users that installed Python 3.13t are facing the reverse issue and being forced to change to 3.13.

Steps to Replicate

  1. Create a new environment
  2. Use the command conda install python
  3. Try to use conda update --all or alternatively something like conda install numpy
  4. It will prompt you to downgrade from Python 3.13.0-hf5aa216_100_cp313 --> 3.13.0-h4077693_0_cp313t

Expected Behavior

When updating all packages or installing a new package on an environment that has the default version of Python with the GIL enabled, respect the distribution of Python installed and do not force the user to move to the free-threaded version.

Other

If you accept the above, and you are not aware that you are using the free-threaded version of Python, you can run into some problems when installing packages with pip. For example, when installing pandas, it will try to reinstall NumPy, but will complain that "python313.lib" does not exist. When you look up, you'll realize it is actually "python313t.lib" that lives there, and many packages aren't ready for that yet. So, moving between different Python distributions when updating packages can be quite a big issue.

Screenshots

These screenshots further show what is happening: Image Image

Installed packages

Name Version Build Channel

bzip2 1.0.8 h2466b09_7 conda-forge ca-certificates 2024.9.24 haa95532_0 anaconda intelpython 2025.0.0 1 https://software.repos.intel.com/python/conda libexpat 2.6.3 he0c23c2_0 conda-forge libffi 3.4.4 hd77b12b_1 anaconda libmpdec 4.0.0 h2466b09_0 conda-forge libsqlite 3.47.0 h2466b09_1 conda-forge libzlib 1.3.1 h2466b09_2 conda-forge numpy 2.1.2 pypi_0 pypi openssl 3.3.2 h2466b09_0 conda-forge pip 24.3.1 pyh145f28c_0 conda-forge python 3.13.0 hf5aa216_100_cp313 conda-forge python_abi 3.13 5_cp313 conda-forge tk 8.6.13 h5226925_1 conda-forge tzdata 2024b hc8b5060_0 conda-forge ucrt 10.0.22621.0 h57928b3_1 conda-forge vc 14.40 h2eaa2aa_1 anaconda vc14_runtime 14.40.33810 hcc2c482_22 conda-forge vs2015_runtime 14.40.33810 h3bf8584_22 conda-forge xz 5.4.6 h8cc25b3_1 anaconda

Environment info

 active environment : dev
active env location : C:\Users\REDACTED\miniforge3\envs\dev
        shell level : 1
   user config file : C:\Users\REDACTED\.condarc

populated config files : C:\Users\REDACTED\miniforge3.condarc C:\Users\REDACTED.condarc conda version : 24.9.2 conda-build version : not installed python version : 3.12.7.final.0 solver : libmamba (default) virtual packages : archspec=1=skylake conda=24.9.2=0 __win=0=0 base environment : C:\Users\REDACTED\miniforge3 (writable) conda av data dir : C:\Users\REDACTED\miniforge3\etc\conda conda av metadata url : None channel URLs : https://conda.anaconda.org/conda-forge/win-64 https://conda.anaconda.org/conda-forge/noarch https://software.repos.intel.com/python/conda/win-64 https://software.repos.intel.com/python/conda/noarch https://conda.anaconda.org/anaconda/win-64 https://conda.anaconda.org/anaconda/noarch https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch package cache : C:\Users\REDACTED\miniforge3\pkgs C:\Users\REDACTED.conda\pkgs C:\Users\REDACTED\AppData\Local\conda\conda\pkgs envs directories : C:\Users\REDACTED\miniforge3\envs C:\Users\REDACTED.conda\envs C:\Users\REDACTED\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/24.9.2 requests/2.32.3 CPython/3.12.7 Windows/11 Windows/10.0.26100 solver/libmamba conda-libmamba-solver/24.9.0 libmambapy/1.5.9 administrator : False netrc file : None offline mode : False

rgommers commented 3 weeks ago

I can reproduce this on macOS arm64 with conda 24.7.1:

$ conda create -n pybug python
$ conda activate pybug
$ conda list python
# packages in environment at /Users/rgommers/mambaforge/envs/pybug:
#
# Name                    Version                   Build  Channel
python                    3.13.0          h75c3a9f_100_cp313    conda-forge
python_abi                3.13                    5_cp313    conda-forge

$ conda update --all
...
## Package Plan ##

  environment location: /Users/rgommers/mambaforge/envs/pybug

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-3.13.0              |hf5be2b4_0_cp313t        12.9 MB  conda-forge
    ------------------------------------------------------------
                                           Total:        12.9 MB

The following packages will be DOWNGRADED:

  python                          3.13.0-h75c3a9f_100_cp313 --> 3.13.0-hf5be2b4_0_cp313t
  python_abi                                   3.13-5_cp313 --> 3.13-5_cp313t

It does say "downgraded" because free-threaded builds have a lower build number than default (with-gil) builds, implemented in commit 94fb486d3322. It feels like this is a known conda bug, perhaps only for older conda versions (@isuruf do you know if this is the case?). mamba 1.5.8 does not have this problem.

isuruf commented 3 hours ago

I'm not sure. Freethreading builds are weighed down by track-features and build numbers, but they are ignored when you do conda update --all to get a 'up-to-date` build. However it looks like conda thinks both are downgrades and still proceeds.

yt87 commented 3 hours ago

What I found with 

mamba --version
mamba 1.5.10
conda 24.9.2
mamba create -n pybug python
mamba activate pybug
mamba update --all

all work fine. Then

mamba install cython is OK

mamba install numpy wants to change python to free-threaded (not downgrade, though).

I have two versions of numpy-2.1.3 in ~/miniforge/pkgs: numpy-2.1.3-py313h4bf6692_0 and numpy-2.1.3-py313hb01392b_0. The latter is the free-threaded one, happens to be higher in lexicographic order. Which may explain the observed behaviour.

yt87 commented 2 hours ago

@isuruf, you are right, of course, except that the upgrade/downgrade message depends on hash. It can go either way. Add a tag to package name? I am sure there must have a discussion about this issue. I guess I will stick with python 3.12 for now.

I realised the tag idea was stupid and deleted it from my post, thinking nobody would notice. You did.