search

georgian-io-archive / foreshadow

An automatic machine learning system
https://foreshadow.readthedocs.io
Apache License 2.0
29 stars 2 forks source link

Potential dependency conflicts between foreshadow and numpy #220

Open NeolithEra opened 4 years ago

NeolithEra commented 4 years ago

Hi, as shown in the following full dependency graph of foreshadow, foreshadow requires numpy >=1.16.4,<2.0.0, foreshadow requires TPOT >=0.11.0,<0.12.0 (TPOT 0.11.1 will be installed, i.e., the newest version satisfying the version constraint), and directed dependency TPOT 0.11.1 transitively introduces numpy >=1.16.3.

Obviously, there are multiple version constraints set for numpy in this project. However, according to pip's “first found wins” installation strategy, numpy 1.18.4 (i.e., the newest version satisfying constraint >=1.16.4,<2.0.0) is the actually installed version.

Although the first found package version numpy 1.18.4 just satisfies the later dependency constraint （numpy >=1.16.3), such installed version is very close to the upper bound of the version constraint of Numpy specified by TPOT 0.11.1.

Once TPOT upgrades，its newest version will be installed. Therefore, it will easily cause a dependency conflict (build failure), if the upgraded TPOT version introduces a higher version of Numpy, violating its another version constraint >=1.16.4,<2.0.0.

According to the release history of TPOT, it habitually upgrates Numpy in its recent releases. For instance, TPOT 0.9.5 upgrated Numpy’s constraint from *_ to >=1.12.1_, and TPOT 0.11.0 upgrated Numpy’s constraint from >=1.12.1 to >=1.16.3**.

As such, it is a warm warning of a potential dependency conflict issue for foreshadow.

Dependency tree

foreshadow - 1.0.1
| +- category-encoders(install version:1.3.0 version range:>=1.2.8,<2.0.0)
| | +- numpy(install version:1.18.4 version range:>=1.11.1)
| | +- pandas(install version:0.25.3 version range:>=0.20.1)
| | +- patsy(install version:0.5.1 version range:>=0.4.1)
| | | +- numpy(install version:1.18.4 version range:>=1.4)
| | | +- six(install version:1.14.0 version range:*)
| | +- scikit-learn(install version:0.22.2.post1 version range:>=0.17.1)
| | +- scipy(install version:1.4.1 version range:>=0.17.0)
| | +- statsmodels(install version:0.11.1 version range:>=0.6.1)
| | | +- numpy(install version:1.18.4 version range:>=1.14)
| | | +- pandas(install version:0.25.3 version range:>=0.21)
| | | +- patsy(install version:0.5.1 version range:>=0.5)
| | | | +- numpy(install version:1.18.4 version range:>=1.4)
| | | | +- six(install version:1.14.0 version range:*)
| | | +- scipy(install version:1.4.1 version range:>=1.0)
| +- fancyimpute(install version:0.3.2 version range:>=0.3.2,<0.4.0)
| +- hyperopt(install version:0.1.2 version range:>=0.1.2,<0.2.0)
| +- jsonpickle(install version:1.4.1 version range:>=1.2,<2.0)
| +- marshmallow(install version:2.21.0 version range:>=2.19.5,<3.0.0)
| +- numpy(install version:1.18.4 version range:>=1.16.4,<2.0.0)
| +- pandas(install version:0.25.3 version range:>=0.25.0,<0.26.0)
| +- patchy(install version:1.5.0 version range:>=1.5,<2.0)
| | +- six(install version:1.14.0 version range:>=1.9.0)
| +- pyyaml(install version:5.3.1 version range:>=5.1,<6.0)
| +- scikit-learn(install version:0.22.2.post1 version range:>=0.22.1,<0.23.0)
| +- scipy(install version:1.4.1 version range:>=1.1.0,<2.0.0)
| +- scs(install version:2.1.0 version range:<=2.1.0)
| | +- numpy(install version:1.18.4 version range:>=1.7)
| | +- scipy(install version:1.4.1 version range:>=0.13.2)
| +- toml(install version:0.10.0 version range:>=0.10.0,<0.11.0)
| +- TPOT(install version:0.11.1 version range:>=0.11.0,<0.12.0)
| | +- deap(install version:1.3.1 version range:>=1.2)
| | | +- numpy(install version:1.18.4 version range:*)
| | +- joblib(install version:0.14.1 version range:>=0.13.2)
| | +- numpy(install version:1.18.4 version range:>=1.16.3)
| | +- pandas(install version:0.25.3 version range:>=0.24.2)
| | +- scikit-learn(install version:0.22.2.post1 version range:>=0.22.0)
| | +- scipy(install version:1.4.1 version range:>=1.3.1)
| | +- stopit(install version:1.1.2 version range:>=1.1.1)
| | +- tqdm(install version:4.45.0 version range:>=4.36.1)
| | +- update-checker(install version:0.16 version range:>=0.16)
| | | +- requests(install version:2.23.0 version range:>=2.3.0)
| | | | +- certifi(install version:2020.4.5.1 version range:>=2017.4.17)
| | | | +- chardet(install version:3.0.4 version range:>=3.0.2,<4)
| | | | +- idna(install version:2.9 version range:>=2.5,<3)
| | | | +- urllib3(install version:1.25.9 version range:>=1.21.1,<1.26)

Thanks for your help. Best, Neolith

NeolithEra commented 4 years ago

Suggested Solution

  1. Loosen the version range of numpy to be >=1.16.4.
  2. Remove your direct dependency numpy, and use the numpy transitively introduced by TPOT.
  3. Change your direct dependency TPOT to be >=0.11.0,<=0.11.1. @adithyabsk Which solution do you prefer, 1 ,2or 3? Please let me know your choice. May I pull a request to solve this issue?
adithyabsk commented 4 years ago

I'm no longer the primary maintainer of the project but I've added the right person to the thread. cc: @JingJZ160