search

conda-forge / pymc-feedstock

A conda-smithy repository for pymc.
BSD 3-Clause "New" or "Revised" License
1 stars 12 forks source link

`No module named 'distutils.msvccompiler'` in fresh environments #58

Closed michaelosthege closed 2 years ago

michaelosthege commented 2 years ago

This is a new problem that was already reported twice on discourse:

The underlying cause is further down the dependency stack, but it's highly likely that we'll need to make some patches in the feedstock until the actual bugfixes become available.

One of the main discussions appears to be:

At the PyMC level, the problem can be reproduced by creating a new environment:

mamba create -n pmtest -c conda-forge "python=3.9" "pymc==4.1.4" -y
mamba activate pmtest
python -c "import pymc"

It looks like in the pyodide issue they've already identified the problem.

The TODO for us remains to investigate which dependencies we have to pin to which versions.

maresb commented 2 years ago

Does this bug only happen for Windows environments?

Does adding "setuptools !=65.*" to the conda/mamba commands fix the issue? For example:

mamba install -c conda-forge "setuptools !=65.*"
random-walks commented 2 years ago

Failing for me on windows even while using

mamba install -c conda-forge "setuptools !=65.*"

running python 3.10

jonititan commented 2 years ago

So I'm on windows 10 and the following suggested by @maresb worked for me. I just did it in a clean environment. conda create -c conda-forge -n pymc_env "setuptools !=65.*" "pymc>=4.0.0"

Edited to add more details Python 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 06:57:50) [MSC v.1929 64 bit (AMD64)] on win32 It gave me pymc v 4.1.4 setuptools v 64.0.3 distutils v 3.10.5

jonititan commented 2 years ago

There is now also a more general thread to cover this. Setuptools Issue 3532 This was caused by Setuptools PR 3505

random-walks commented 2 years ago

Thanks @jonititan and @maresb, working now building the fresh env using that command.

fingers crossed on a patch soon looks like this bug is wreaking some havoc on a bunch of pkgs

jonititan commented 2 years ago

It looks like the breaking change in setuptools is now also being reverted so that should flow through in due course . https://github.com/pypa/setuptools/pull/3505#issuecomment-1217210712

maresb commented 2 years ago

The new setuptools version 65.0.2 which reverts the breaking change has been merged into conda-forge, and is already available for download.

As far as I can tell, this can be triggered in the following two ways:

  1. by PyMC directly via my patch to suppress spurious warnings, which affects PyMC v4.1.2, v4.1.3, and v4.1.4. (Subsequent versions will be fine due to https://github.com/pymc-devs/pymc/pull/6025.)
  2. indirectly by aesara >=2.5.0,<=2.7.7.

I intend to fix 2. by retroactively adding a setuptools !=65 to https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/pull/307.

We have the following dependencies of PyMC on Aesara:

Thus by fixing 2, the only broken version will be PyMC v4.1.4. This should be fixable by a repodata patch.

Since repodata patches are annoying and time-consuming, the most practical solution is to extend the setuptools !=65patch to aesara >=2.5.0,<=2.7.7. This is slightly more than technically necessary, but is an elegant solution. (Otherwise I'd have to do another repodata patch just for PyMC v4.1.4.)

maresb commented 2 years ago

To be explicit, this should now be effectively fixed by the upstream release of setuptools v65.0.2.

New environments should default to the latest setuptools v65.0.2. Old broken environments should be fixable via the command above.

The proposed solution in my previous comment takes care of any remaining edge cases by actively preventing the solver from installing a broken version setuptools alongside PyMC or Aesara.

maresb commented 2 years ago

@random-walks, could you please open a new issue of type "Bug report" and fill in the template with the details of your environment so that we can have a closer look at what's going on?

maresb commented 2 years ago

I'm waiting for https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/pull/307 to be merged before closing this. They seem to have a backlog of non-critical PRs. This should be fixed at some point, but I don't see it as urgent, so I'm going to be patient.

maresb commented 2 years ago

Closed via https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/pull/307