problem installing songbird 1.0.2 with qiime2-2020.2

[gwarmstrong]

So currently it seems the conda-forge build of songbird=1.0.2 and qiime2-2020.2 are unsatisfiable.

Steps to recreate

$ wget https://data.qiime2.org/distro/core/qiime2-2020.2-py36-osx-conda.yml
$ conda env create -n qiime2-2020.2-songbird --file qiime2-2020.2-py36-osx-conda.yml
$ conda activate qiime2-2020.2-songbird
$ conda install "songbird=1.0.2" -c conda-forge
Collecting package metadata (current_repodata.json): done
Solving environment: / conda install "songbird=1.0.2" -c conda-forge
Collecting package metadata (current_repodatafailed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Package ipython_genutils conflicts for:
songbird=1.0.2 -> ipython -> traitlets[version='>=4.2'] -> ipython_genutils
Package python_abi conflicts for:
python=3.6.7 -> pip -> setuptools -> python_abi[version='3.6|3.6.*',build='*_cp36m|*_pypy36_pp73']
songbird=1.0.2 -> ipython -> python_abi=3.7[build=*_cp37m]
Package certifi conflicts for:
songbird=1.0.2 -> scikit-bio[version='>0.5.4'] -> cachecontrol[version='>=0.11.5'] -> requests -> urllib3[version='>=1.21.1,<1.24'] -> certifi
python=3.6.7 -> pip -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']
songbird=1.0.2 -> scikit-bio[version='>0.5.4'] -> cachecontrol[version='>=0.11.5'] -> requests -> certifi[version='>=2017.4.17']
Package setuptools conflicts for:
python=3.6.7 -> pip -> setuptools
songbird=1.0.2 -> tensorflow==1.14.0 -> tensorboard[version='>=1.14.0,<1.15.0'] -> markdown[version='>=2.6.8'] -> setuptools[version='>=36']
songbird=1.0.2 -> nose[version='>=1.3.7'] -> setuptools[version='>=18.5']
songbird=1.0.2 -> scikit-bio[version='>0.5.4'] -> pytest -> setuptools[version='>=40.0']
Package python conflicts for:
python=3.6.7
Package ca-certificates conflicts for:
songbird=1.0.2 -> ipython -> setuptools[version='>=18.5'] -> ca-certificates
python=3.6.7 -> openssl[version='>=1.1.1a,<1.1.2a'] -> ca-certificates
Package ipython conflicts for:
songbird=1.0.2 -> mock -> ipython[version='>=3.2.0']
songbird=1.0.2 -> ipython

[fedarko]

Just noting for anyone else who runs into this problem -- as a band-aid solution, you can install songbird via pip and that should circumvent this problem.

edit: As a more lasting solution, Songbird's conda-forge recipe should be updated: both to be consistent with the setup.py stuff, and to remove unneeded things (I don't think ipython is explicitly required by Songbird, for example). See #45.

[fedarko]

@gwarmstrong when you get a chance could you check that the new conda recipe fixes the problem for you?

[fedarko]

Update, I got around to testing the newest recipe on Q2 2020.2. It now installs correctly (:D), but I got a weird "version conflict" error about TensorFlow -- this was resolvable by just pip-installing TensorFlow 1.15.2 directly.

TLDR, this error (which is different from the ones documented earlier in this thread ._.) happens because the new conda-forge recipe installs TF v1.15.0 -- this is the latest 1.x version of TF on conda that I could find. This is as expected, but it looks like Python/conda/etc. detects that songbird's setup.py asks for TF >= 1.15.2 (because we can do that on pip, since this version is on pip and earlier versions of TF 1.15.x have a small security problem that was fixed in 1.15.2) and then notices that 1.15.0 is less than 1.15.2 and then explodes.

So the way to fix this new error long-term would be downgrading the minimum TF version in setup.py, but of course this would miss out on the security fixes from 1.15.2. Ugh.

(in any case, things are easier to handle now)

[gwarmstrong]

Thanks for all the work on this!! Sorry I hadn’t gotten around to trying it yet. Wait, why not pin the tf version to >=1.15.2 in the conda build?

[fedarko]

I think doing that will cause different problems, since TF version >= 1.15.2, <2 isn't installable at all through conda at present. Leaving out the , <2 part of that will get rid of the errors (yay) and tell conda to install TF 2.x (:thinking:), which should work with songbird but may cause other problems... hence the struggle.

[gwarmstrong]

Update, I got around to testing the newest recipe on Q2 2020.2. It now installs correctly (:D), but I got a weird "version conflict" error about TensorFlow -- this was resolvable by just pip-installing TensorFlow 1.15.2 directly.

TLDR, this error (which is different from the ones documented earlier in this thread ._.) happens because the new conda-forge recipe installs TF v1.15.0 -- this is the latest 1.x version of TF on conda that I could find. This is as expected, but it looks like Python/conda/etc. detects that songbird's setup.py asks for TF >= 1.15.2 (because we can do that on pip, since this version is on pip and earlier versions of TF 1.15.x have a small security problem that was fixed in 1.15.2) and then notices that 1.15.0 is less than 1.15.2 and then explodes.

So the way to fix this new error long-term would be downgrading the minimum TF version in setup.py, but of course this would miss out on the security fixes from 1.15.2. Ugh.

(in any case, things are easier to handle now)

Was able to recreate this.