Qiime2-2024.5 and empress - scikit-bio issue

ARW-UBT commented 1 week ago

Hello, I was using empress as a visualizer for phylogenetic trees produced in qiime2 releases up to v. 2024.2. It seems that empress is to some extent incompatible to q2-2024.5 releases as it downgrades scikit-bio-0.5.9 scipy-1.10.1 to previous versions. A post has been opend in q2 forum: https://forum.qiime2.org/t/qiime2-library-compatibility/31620

Since I am not sure whether you are aware of this issue, I would like to open this issue here and ask if you can offer a solution here?

Thanks again for you great tree viewer! Best,

fedarko commented 1 week ago

This is news to me -- thank you for bringing it up. It is strange that installing empress downgrades these packages, since its requirements do not impose any explicit restrictions on the versions of scikit-bio or scipy:

https://github.com/biocore/empress/blob/cc3975a71c931f2d356856fb5ee9669b3f83f150/setup.py#L44-L48

This suggests to me that something is up with one of its dependencies.

To test, I installed a fresh QIIME 2 2024.5 environment, then installed empress into this environment. I observed the same downgrades you mention -- from scikit-bio 0.6.0 to 0.5.9, and from scipy 1.13.0 to 1.10.1.

Log 1: downgrading

marcus@gamma ~> which pip                                                                                                                                                                              (q2-2024.5) 
/home/marcus/anaconda3/envs/q2-2024.5/bin/pip
marcus@gamma ~> pip install empress                                                                                                                                                                    (q2-2024.5) 
Collecting empress
  Using cached empress-1.2.0.tar.gz (302 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.26.4)
Requirement already satisfied: scipy in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.13.0)
Requirement already satisfied: pandas in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (2.2.2)
Requirement already satisfied: click in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (8.1.7)
Requirement already satisfied: jinja2 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (3.1.4)
Requirement already satisfied: scikit-bio in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (0.6.0)
Requirement already satisfied: biom-format in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (2.1.15)
Requirement already satisfied: iow in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.0.5)
Requirement already satisfied: emperor>=1.0.2 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.0.4)
Requirement already satisfied: future in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from emperor>=1.0.2->empress) (1.0.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from jinja2->empress) (2.1.5)
Requirement already satisfied: requests>=2.20.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (2.32.3)
Requirement already satisfied: decorator>=3.4.2 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (4.4.2)
Requirement already satisfied: natsort>=4.0.3 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (8.4.0)
Requirement already satisfied: h5py>=3.6.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (3.11.0)
Requirement already satisfied: hdmedians>=0.14.1 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (0.14.2)
Requirement already satisfied: python-dateutil>=2.8.2 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->empress) (2.9.0)
Requirement already satisfied: pytz>=2020.1 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->empress) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->empress) (2024.1)
Collecting nose>=1.3.7 (from iow->empress)
  Using cached nose-1.3.7-py3-none-any.whl.metadata (1.7 kB)
Collecting cython>=0.24.1 (from iow->empress)
  Using cached Cython-3.0.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
Collecting scikit-bio (from empress)
  Using cached scikit-bio-0.5.9.tar.gz (3.5 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: IPython>=3.2.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (8.18.1)
Requirement already satisfied: matplotlib>=1.4.3 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (3.8.4)
Collecting scipy (from empress)
  Downloading scipy-1.10.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (58 kB)
Requirement already satisfied: jedi>=0.16 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (0.19.1)
Requirement already satisfied: matplotlib-inline in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (0.1.7)
Requirement already satisfied: prompt-toolkit<3.1.0,>=3.0.41 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (3.0.47)
Requirement already satisfied: pygments>=2.4.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (2.18.0)
Requirement already satisfied: stack-data in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (0.6.2)
Requirement already satisfied: traitlets>=5 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (5.9.0)
Requirement already satisfied: typing-extensions in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (4.12.2)
Requirement already satisfied: exceptiongroup in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (1.2.2)
Requirement already satisfied: pexpect>4.3 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->empress) (4.9.0)
Requirement already satisfied: contourpy>=1.0.1 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (1.3.0)
Requirement already satisfied: cycler>=0.10 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (1.4.7)
Requirement already satisfied: packaging>=20.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (24.1)
Requirement already satisfied: pillow>=8 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (10.3.0)
Requirement already satisfied: pyparsing>=2.3.1 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (3.1.4)
Requirement already satisfied: importlib-resources>=3.2.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from matplotlib>=1.4.3->scikit-bio->empress) (6.4.4)
Requirement already satisfied: six>=1.5 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->empress) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (2024.8.30)
Requirement already satisfied: zipp>=3.1.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib>=1.4.3->scikit-bio->empress) (3.20.1)
Requirement already satisfied: parso<0.9.0,>=0.8.3 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from jedi>=0.16->IPython>=3.2.0->scikit-bio->empress) (0.8.4)
Requirement already satisfied: ptyprocess>=0.5 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pexpect>4.3->IPython>=3.2.0->scikit-bio->empress) (0.7.0)
Requirement already satisfied: wcwidth in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from prompt-toolkit<3.1.0,>=3.0.41->IPython>=3.2.0->scikit-bio->empress) (0.2.13)
Requirement already satisfied: executing>=1.2.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from stack-data->IPython>=3.2.0->scikit-bio->empress) (2.1.0)
Requirement already satisfied: asttokens>=2.1.0 in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from stack-data->IPython>=3.2.0->scikit-bio->empress) (2.4.1)
Requirement already satisfied: pure-eval in ./anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from stack-data->IPython>=3.2.0->scikit-bio->empress) (0.2.3)
Downloading scipy-1.10.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.5/34.5 MB 4.7 MB/s eta 0:00:00
Using cached Cython-3.0.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
Using cached nose-1.3.7-py3-none-any.whl (154 kB)
Building wheels for collected packages: empress, scikit-bio
  Building wheel for empress (setup.py) ... done
  Created wheel for empress: filename=empress-1.2.0-py3-none-any.whl size=294959 sha256=d4830380a22d932ae946c7ef31cd105cb9efa83a70dc16765f3a565a59d53ede
  Stored in directory: /home/marcus/.cache/pip/wheels/4b/c6/c4/55274e58e94e677bb8db5ca295d90786fce9987873657296f9
  Building wheel for scikit-bio (pyproject.toml) ... done
  Created wheel for scikit-bio: filename=scikit_bio-0.5.9-cp39-cp39-linux_x86_64.whl size=1271087 sha256=3b5f6b0fc718e628842388fb251a4600a4320e9510890b0124b7784f3a52af67
  Stored in directory: /home/marcus/.cache/pip/wheels/01/4b/87/799e0e29e168e76cf7c49bacc229116c053a002c79b9f2486c
Successfully built empress scikit-bio
Installing collected packages: nose, scipy, cython, scikit-bio, empress
  Attempting uninstall: scipy
    Found existing installation: scipy 1.13.0
    Uninstalling scipy-1.13.0:
      Successfully uninstalled scipy-1.13.0
  Attempting uninstall: scikit-bio
    Found existing installation: scikit-bio 0.6.0
    Uninstalling scikit-bio-0.6.0:
      Successfully uninstalled scikit-bio-0.6.0
Successfully installed cython-3.0.11 empress-1.2.0 nose-1.3.7 scikit-bio-0.5.9 scipy-1.10.1

This log confirmed that I could reproduce the problem, but it was not very helpful for debugging. What helped was, after the downgrades, trying to upgrade the packages in question, and then seeing what pip complained about:

Log 2: trying to upgrade scikit-bio and seeing what happens

marcus@gamma ~/D/W/K/E/f/empress (fix-562) [1]> pip install scikit-bio==0.6.0                                                                                                                          (q2-2024.5) 
Collecting scikit-bio==0.6.0
  Downloading scikit-bio-0.6.0.tar.gz (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 3.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: requests>=2.20.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (2.32.3)
Requirement already satisfied: decorator>=3.4.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (4.4.2)
Requirement already satisfied: natsort>=4.0.3 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (8.4.0)
Requirement already satisfied: numpy>=1.17.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (1.26.4)
Requirement already satisfied: pandas>=1.5.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (2.2.2)
Requirement already satisfied: scipy>=1.9.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (1.10.1)
Requirement already satisfied: h5py>=3.6.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (3.11.0)
Requirement already satisfied: hdmedians>=0.14.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (0.14.2)
Requirement already satisfied: biom-format in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio==0.6.0) (2.1.15)
Requirement already satisfied: Cython>=0.23 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from hdmedians>=0.14.1->scikit-bio==0.6.0) (3.0.11)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas>=1.5.0->scikit-bio==0.6.0) (2.9.0)
Requirement already satisfied: pytz>=2020.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas>=1.5.0->scikit-bio==0.6.0) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas>=1.5.0->scikit-bio==0.6.0) (2024.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio==0.6.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio==0.6.0) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio==0.6.0) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio==0.6.0) (2024.8.30)
Requirement already satisfied: click in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from biom-format->scikit-bio==0.6.0) (8.1.7)
Requirement already satisfied: six>=1.5 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas>=1.5.0->scikit-bio==0.6.0) (1.16.0)
Building wheels for collected packages: scikit-bio
  Building wheel for scikit-bio (pyproject.toml) ... done
  Created wheel for scikit-bio: filename=scikit_bio-0.6.0-cp39-cp39-linux_x86_64.whl size=1474918 sha256=1c1954de432790e8b446c3b7ee0648f5c837959cd24e42d37a7b100a95842c52
  Stored in directory: /home/marcus/.cache/pip/wheels/e1/85/a9/90472cfca80499f8f384f59a6ab2759a93e259ff0b00937267
Successfully built scikit-bio
Installing collected packages: scikit-bio
  Attempting uninstall: scikit-bio
    Found existing installation: scikit-bio 0.5.9
    Uninstalling scikit-bio-0.5.9:
      Successfully uninstalled scikit-bio-0.5.9
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
deblur 1.1.1 requires scikit-bio<0.6.0,>=0.5.0, but you have scikit-bio 0.6.0 which is incompatible.
iow 1.0.5 requires scikit-bio<0.6.0,>=0.5.0, but you have scikit-bio 0.6.0 which is incompatible.
Successfully installed scikit-bio-0.6.0

Looks like the problematic packages here are deblur and iow.

I am not sure what's up with deblur (version 1.1.1 of deblur is already what Q2 2024.5 uses -- also, I don't think empress or any of its dependencies rely on it?) So I think that is unrelated.
However, the iow 1.0.5 version used in Q2 2024.5 seems problematic. There have since been two new versions of iow released, the most recent of which (1.0.7) removed a pin on the scikit-bio version.

Solution: upgrading iow

As far as I can tell, the cause of the downgrades is iow 1.0.5. To test this, I again set up a fresh Q2 2024.5 environment, then upgraded iow 1.0.5 to iow 1.0.7 within that environment (pip install iow==1.0.7), then verified that installing empress into this environment did not cause any downgrades (see logs below).

I am not sure of the implications of upgrading iow 1.0.5 to 1.0.7, but it does not look like it should break anything (copying @wasade for reference). Hopefully, future versions of QIIME 2 will use the fixed version of iow, which should resolve this particular problem. (But for now, I think just upgrading iow as shown above before installing empress should fix the problem. You can probably also just manually upgrade scikit-bio / scipy to fix an existing Q2 environment, also.)

Log 3: upgrading iow, then installing empress

marcus@gamma ~/D/W/K/E/f/empress (fix-562)> pip install iow==1.0.7                                                                                                                                     (q2-2024.5) 
Collecting iow==1.0.7
  Using cached iow-1.0.7-cp39-cp39-linux_x86_64.whl
Requirement already satisfied: numpy>=1.9.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from iow==1.0.7) (1.26.4)
Collecting nose>=1.3.7 (from iow==1.0.7)
  Using cached nose-1.3.7-py3-none-any.whl.metadata (1.7 kB)
Collecting cython<1.0.0,>=0.24.1 (from iow==1.0.7)
  Using cached Cython-0.29.37-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (3.1 kB)
Requirement already satisfied: pandas in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from iow==1.0.7) (2.2.2)
Requirement already satisfied: click in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from iow==1.0.7) (8.1.7)
Requirement already satisfied: scikit-bio>=0.5.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from iow==1.0.7) (0.6.0)
Requirement already satisfied: requests>=2.20.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio>=0.5.0->iow==1.0.7) (2.32.3)
Requirement already satisfied: decorator>=3.4.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio>=0.5.0->iow==1.0.7) (4.4.2)
Requirement already satisfied: natsort>=4.0.3 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio>=0.5.0->iow==1.0.7) (8.4.0)
Requirement already satisfied: scipy>=1.9.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio>=0.5.0->iow==1.0.7) (1.13.0)
Requirement already satisfied: h5py>=3.6.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio>=0.5.0->iow==1.0.7) (3.11.0)
Requirement already satisfied: hdmedians>=0.14.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio>=0.5.0->iow==1.0.7) (0.14.2)
Requirement already satisfied: biom-format in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio>=0.5.0->iow==1.0.7) (2.1.15)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->iow==1.0.7) (2.9.0)
Requirement already satisfied: pytz>=2020.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->iow==1.0.7) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->iow==1.0.7) (2024.1)
Requirement already satisfied: six>=1.5 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->iow==1.0.7) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio>=0.5.0->iow==1.0.7) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio>=0.5.0->iow==1.0.7) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio>=0.5.0->iow==1.0.7) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio>=0.5.0->iow==1.0.7) (2024.8.30)
Using cached Cython-0.29.37-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
Using cached nose-1.3.7-py3-none-any.whl (154 kB)
Installing collected packages: nose, cython, iow
  Attempting uninstall: iow
    Found existing installation: iow 1.0.5
    Uninstalling iow-1.0.5:
      Successfully uninstalled iow-1.0.5
Successfully installed cython-0.29.37 iow-1.0.7 nose-1.3.7
marcus@gamma ~/D/W/K/E/f/empress (fix-562)> pip install empress                                                                                                                                        (q2-2024.5) 
Collecting empress
  Using cached empress-1.2.0-py3-none-any.whl
Requirement already satisfied: numpy in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.26.4)
Requirement already satisfied: scipy in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.13.0)
Requirement already satisfied: pandas in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (2.2.2)
Requirement already satisfied: click in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (8.1.7)
Requirement already satisfied: jinja2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (3.1.4)
Requirement already satisfied: scikit-bio in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (0.6.0)
Requirement already satisfied: biom-format in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (2.1.15)
Requirement already satisfied: iow in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.0.7)
Requirement already satisfied: emperor>=1.0.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from empress) (1.0.4)
Requirement already satisfied: future in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from emperor>=1.0.2->empress) (1.0.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from jinja2->empress) (2.1.5)
Requirement already satisfied: requests>=2.20.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (2.32.3)
Requirement already satisfied: decorator>=3.4.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (4.4.2)
Requirement already satisfied: natsort>=4.0.3 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (8.4.0)
Requirement already satisfied: h5py>=3.6.0 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (3.11.0)
Requirement already satisfied: hdmedians>=0.14.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from scikit-bio->empress) (0.14.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->empress) (2.9.0)
Requirement already satisfied: pytz>=2020.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->empress) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from pandas->empress) (2024.1)
Requirement already satisfied: nose>=1.3.7 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from iow->empress) (1.3.7)
Requirement already satisfied: cython<1.0.0,>=0.24.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from iow->empress) (0.29.37)
Requirement already satisfied: six>=1.5 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->empress) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /home/marcus/anaconda3/envs/q2-2024.5/lib/python3.9/site-packages (from requests>=2.20.0->scikit-bio->empress) (2024.8.30)
Installing collected packages: empress
Successfully installed empress-1.2.0

ARW-UBT commented 1 week ago

Hi @fedarko Thank you for the workaround! I was able to apply it to my installations (tested it on two different machines, I had to install Cython first, as this was missing), and I was able to install empress without any issues. The empress phylogenetic tree is as I knew it from q2-2024.2 versions.

Since the same issue occured also with q2-gemelli, I installed gemelli after iow update, and this worked now also without any downgrade issues for scipy and scikit-bio. see https://github.com/biocore/gemelli/issues/104

Thank you again for your quick help! Best,

wasade commented 1 week ago

iow is not compatible with Cython 3 and the pins reflect that. I would recommend using 1.0.7 as it is numpy 2.0 compatible, and works with skbio >= 0.6.0

ARW-UBT commented 2 days ago

Hi @wasade and @fedarko I just installed q2-amplicon-2024.10 in a new conda environment and followed your workaround. Since the q2-2024.10 version installs iow-1.0.5, I did an update for iow and tried avoid incompatibilities with Cython 3 by the following commands (using the Cython version from the workaround above): pip install Cython==0.29.37 iow==1.0.7

Then I installed empress and gemelli pip install empress pip install gemelli

No problems so far :-) If anyone will apply the same workaround, I would very much appreciate a short comment to be sure that no other problems have been introduced.

Best,

biocore / empress