search

kmayerb / tcrdist3

flexible CDR based distance metrics
MIT License
53 stars 17 forks source link

Installation files not found #67

Closed wukevin closed 2 years ago

wukevin commented 2 years ago

I am running into issues when trying to install tcrdist3 using a conda environment. Here are the steps to reproduce, along with the resulting error:

> conda create --name tcrdist3 python=3.8
> conda activate tcrdist3
> pip install tcrdist3
> pip install notebook ipython

All the above execute without error. The resulting error: 

❯ python
Python 3.8.12 | packaged by conda-forge | (default, Jan 30 2022, 23:53:36)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from tcrdist.repertoire import TCRrep
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wukevin/projects/tcr/tcr-dist/tcrdist/__init__.py", line 11, in <module>
    from . import processing
  File "/home/wukevin/projects/tcr/tcr-dist/tcrdist/processing.py", line 4, in <module>
    from .blast import parse_unpaired_dna_sequence_blastn, get_qualstring
  File "/home/wukevin/projects/tcr/tcr-dist/tcrdist/blast.py", line 7, in <module>
    from .paths import path_to_db, path_to_blast_executables
  File "/home/wukevin/projects/tcr/tcr-dist/tcrdist/paths.py", line 13, in <module>
    assert op.isdir( path_to_db )
AssertionError
kmayerb commented 2 years ago

Interesting I have never seen this error with tcrdist3 before.

I think you somehow downloaded Phil Bradley's original tcr-dist or tcrdist2, because tcrdist3 does not use blast at all.

Try install again:

conda create --name tcrdist3 python=3.8
conda activate tcrdist3
pip install git+https://github.com/kmayerb/tcrdist3.git@0.2.2
pip install notebook ipython

This way you will be guaranteed to be getting the latest tcrdist3 version in your conda env

I think the other possible problem might be you are in a working directory with an old version of tcr-dist such that when you execute this: 

from tcrdist.repertoire import TCRrep

Perhaps python is looking in the current working directory and not at your python environment. Try again invoking Python, perhaps from your home directory instead.

Let me know if any of these approaches, doesn't solve the issue.

wukevin commented 2 years ago

Thanks! I think I was in a working directory with an old version of tcr-dist, so python was trying to import that instead.

However, I'm now running into a different problem:

❯ python
Python 3.8.12 | packaged by conda-forge | (default, Jan 30 2022, 23:53:36)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from tcrdist.repertoire import TCRrep
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/tcrdist/repertoire.py", line 6, in <module>
    from tcrdist.rep_funcs import _pws, compute_pws_sparse
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/tcrdist/rep_funcs.py", line 11, in <module>
    from tcrdist import memory
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/tcrdist/memory.py", line 8, in <module>
    from hierdiff.tally import neighborhood_tally
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/hierdiff/__init__.py", line 3, in <module>
    from .association_testing import cluster_association_test
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/hierdiff/association_testing.py", line 15, in <module>
    from fishersapi import fishers_vec, fishers_frame, adjustnonnan
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/fishersapi/__init__.py", line 3, in <module>
    from .fishersapi import *
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/fishersapi/fishersapi.py", line 66, in <module>
    import fisher
  File "/home/wukevin/miniconda3/envs/tcrdist3/lib/python3.8/site-packages/fisher/__init__.py", line 3, in <module>
    from .cfisher import *
  File "src/cfisher.pyx", line 1, in init fisher.cfisher
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

It appears that by default, the pip commands install numpy = 1.21.x. This SO thread looks related https://stackoverflow.com/questions/66060487/valueerror-numpy-ndarray-size-changed-may-indicate-binary-incompatibility-exp but downgrading to 1.20.3 as suggested there doesn't help. Have you seen this issue before?

If it is helpful, I'm on Ubuntu:

❯ cat /proc/version
Linux version 5.4.0-96-generic (buildd@lcy02-amd64-007) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #109~18.04.1-Ubuntu SMP Thu Jan 13 15:06:26 UTC 2022
kmayerb commented 2 years ago

see similar closed issue: https://github.com/kmayerb/tcrdist3/issues/66#issuecomment-1011797071

kmayerb commented 2 years ago

Note exactly sure; also, perhaps there is something in the testing that might help you diagnose:

I wonder if this would help, as we do it in the CI testing:

python -m pip install --upgrade pip

From CI test results, looks like 1.21.5 numpy version is used.

https://github.com/kmayerb/tcrdist3/runs/4644590513?check_suite_focus=true

Collecting numpy>=1.18.1 Downloading numpy-1.21.5-cp38-cp38-manylinux2[12](https://github.com/kmayerb/tcrdist3/runs/4644590513?

wukevin commented 2 years ago

That's weird, version 1.21.5 of numpy does not work, and neither does the requirements.txt blob from the issue you referenced: 

appnope==0.1.2
backcall==0.2.0
certifi==2021.10.8
cycler==0.11.0
decorator==5.1.0
dill==0.3.4
feather-format==0.4.1
fisher==0.1.9
fishersapi==0.3
fonttools==4.28.3
hierdiff==0.8
ipython==7.30.1
jedi==0.18.1
Jinja2==3.0.3
joblib==1.1.0
kiwisolver==1.3.2
llvmlite==0.37.0
MarkupSafe==2.0.1
matplotlib==3.5.1
matplotlib-inline==0.1.3
numba==0.54.1
numpy==1.20.3
olga==1.2.4
packaging==21.3
palmotif==0.4
pandas==1.3.5
parasail==1.2.4
parmap==1.5.3
parso==0.8.3
patsy==0.5.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.4.0
progress==1.6
prompt-toolkit==3.0.24
ptyprocess==0.7.0
pwseqdist==0.6
pyarrow==6.0.1
Pygments==2.10.0
pynndescent==0.5.5
pyparsing==3.0.6
python-dateutil==2.8.2
pytz==2021.3
scikit-learn==1.0.1
scipy==1.7.3
seaborn==0.11.2
six==1.16.0
sklearn==0.0
statsmodels==0.13.1
svgwrite==1.4.1
tcrdist3 @ git+https://github.com/kmayerb/tcrdist3.git@04b0b8c2573d04a9d2cb77f7a3aeeed3a0eab167
tcrsampler==0.1.9
threadpoolctl==3.0.0
tqdm==4.62.3
traitlets==5.1.1
umap-learn==0.5.2
wcwidth==0.2.5
zipdist==0.1.5

I'm not sure how much you can diagnose remotely, or what additional information you'd need.

Appreciate all your help with this!

kmayerb commented 2 years ago

Sorry you are having trouble. We test the package regularly using Ubuntu containers, so I am at a loss for an explanation.

Have you tried using the docker container, I bet there is an unusual issue with your local environment.

Also, you could try install without condas using a venv, see youtube video where we demo that here: https://tcrdist3.readthedocs.io/en/latest/index.html#venv

On Thu, Feb 17, 2022 at 5:40 PM Kevin Wu @.***> wrote:

That's weird, version 1.21.5 of numpy does not work, and neither does the requirements.txt blob from the issue you referenced:

appnope==0.1.2 backcall==0.2.0 certifi==2021.10.8 cycler==0.11.0 decorator==5.1.0 dill==0.3.4 feather-format==0.4.1 fisher==0.1.9 fishersapi==0.3 fonttools==4.28.3 hierdiff==0.8 ipython==7.30.1 jedi==0.18.1 Jinja2==3.0.3 joblib==1.1.0 kiwisolver==1.3.2 llvmlite==0.37.0 MarkupSafe==2.0.1 matplotlib==3.5.1 matplotlib-inline==0.1.3 numba==0.54.1 numpy==1.20.3 olga==1.2.4 packaging==21.3 palmotif==0.4 pandas==1.3.5 parasail==1.2.4 parmap==1.5.3 parso==0.8.3 patsy==0.5.2 pexpect==4.8.0 pickleshare==0.7.5 Pillow==8.4.0 progress==1.6 prompt-toolkit==3.0.24 ptyprocess==0.7.0 pwseqdist==0.6 pyarrow==6.0.1 Pygments==2.10.0 pynndescent==0.5.5 pyparsing==3.0.6 python-dateutil==2.8.2 pytz==2021.3 scikit-learn==1.0.1 scipy==1.7.3 seaborn==0.11.2 six==1.16.0 sklearn==0.0 statsmodels==0.13.1 svgwrite==1.4.1 tcrdist3 @ @. tcrsampler==0.1.9 threadpoolctl==3.0.0 tqdm==4.62.3 traitlets==5.1.1 umap-learn==0.5.2 wcwidth==0.2.5 zipdist==0.1.5 @.==0.1.9threadpoolctl==3.0.0tqdm==4.62.3traitlets==5.1.1umap-learn==0.5.2wcwidth==0.2.5zipdist==0.1.5>

I'm not sure how much you can diagnose remotely, or what additional information you'd need.

— Reply to this email directly, view it on GitHub https://github.com/kmayerb/tcrdist3/issues/67#issuecomment-1043713399, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALD2PV5MMVCGDTI6DPA3RNLU3WPRTANCNFSM5OQGB4ZQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

wukevin commented 2 years ago

I've done a bit more testing, it appears that the problem is not specific to Ubuntu; I get nearly the same error when trying to install via pip within the conda environment on my Mac.

That said, the venv does work. Thanks!