Closed shishir-reddy closed 1 year ago
I have added the numpy version I used on the requirements (it was missing for some reason), it is numpy==1.21.5
. I also used python3.7, as I see you use 3.9. My suggestion would be that you create a new environment on python3.7 and re-install the requirements with the new numpy version constraint.
I have created a new environment with python3.7 and the updated requirements.txt file, but I am still seeing some dependency conflicts. It looks like there is a conflict where seqdist 0.0.3 depends on numpy <= 1.18.5:
(venv3) s@bwt:~/mobile_workspace/original/basecalling_architectures$ pip install -r requirements.txt
Collecting numpy==1.21.5 (from -r requirements.txt (line 1))
Using cached numpy-1.21.5-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Collecting cupy-cuda102==9.4.0 (from -r requirements.txt (line 2))
Using cached cupy_cuda102-9.4.0-cp37-cp37m-manylinux1_x86_64.whl (60.7 MB)
Collecting fast_ctc_decode==0.3.0 (from -r requirements.txt (line 3))
Using cached fast_ctc_decode-0.3.0-cp37-cp37m-manylinux2010_x86_64.whl (508 kB)
Collecting h5py==3.7.0 (from -r requirements.txt (line 4))
Using cached h5py-3.7.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.1 MB)
Collecting koi-cuda102==0.0.5 (from -r requirements.txt (line 5))
Using cached koi_cuda102-0.0.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (350 kB)
Collecting ont-fast5-api==4.0.2 (from -r requirements.txt (line 6))
Using cached ont_fast5_api-4.0.2-py3-none-any.whl (2.0 MB)
Collecting pandas==1.3.5 (from -r requirements.txt (line 7))
Using cached pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
Collecting parasail==1.2.4 (from -r requirements.txt (line 8))
Using cached parasail-1.2.4-py2.py3-none-manylinux2010_x86_64.whl (14.1 MB)
Collecting psutil==5.8.0 (from -r requirements.txt (line 9))
Using cached psutil-5.8.0-cp37-cp37m-manylinux2010_x86_64.whl (296 kB)
Collecting scipy==1.7.1 (from -r requirements.txt (line 10))
Using cached scipy-1.7.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.5 MB)
Collecting seqdist==0.0.3 (from -r requirements.txt (line 11))
Using cached seqdist-0.0.3-py3-none-any.whl (21 kB)
Collecting torch==1.9.0 (from -r requirements.txt (line 12))
Using cached torch-1.9.0-cp37-cp37m-manylinux1_x86_64.whl (831.4 MB)
Collecting torchaudio==0.9.0 (from -r requirements.txt (line 13))
Using cached torchaudio-0.9.0-cp37-cp37m-manylinux1_x86_64.whl (1.9 MB)
Collecting torchvision==0.10.0 (from -r requirements.txt (line 14))
Using cached torchvision-0.10.0-cp37-cp37m-manylinux1_x86_64.whl (22.1 MB)
Collecting tqdm==4.62.0 (from -r requirements.txt (line 15))
Using cached tqdm-4.62.0-py2.py3-none-any.whl (76 kB)
Collecting fastrlock>=0.5 (from cupy-cuda102==9.4.0->-r requirements.txt (line 2))
Using cached fastrlock-0.8.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_24_x86_64.whl (46 kB)
Collecting cffi (from koi-cuda102==0.0.5->-r requirements.txt (line 5))
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting packaging (from ont-fast5-api==4.0.2->-r requirements.txt (line 6))
Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting progressbar33>=2.3.1 (from ont-fast5-api==4.0.2->-r requirements.txt (line 6))
Using cached progressbar33-2.4.tar.gz (10 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting python-dateutil>=2.7.3 (from pandas==1.3.5->-r requirements.txt (line 7))
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz>=2017.3 (from pandas==1.3.5->-r requirements.txt (line 7))
Using cached pytz-2023.3-py2.py3-none-any.whl (502 kB)
INFO: pip is looking at multiple versions of seqdist to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 10), -r requirements.txt (line 11), -r requirements.txt (line 2), -r requirements.txt (line 4), -r requirements.txt (line 6), -r requirements.txt (line 7), -r requirements.txt (line 8) and numpy==1.21.5 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested numpy==1.21.5
cupy-cuda102 9.4.0 depends on numpy<1.24 and >=1.17
h5py 3.7.0 depends on numpy>=1.14.5
ont-fast5-api 4.0.2 depends on numpy>=1.16
pandas 1.3.5 depends on numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10"
parasail 1.2.4 depends on numpy
scipy 1.7.1 depends on numpy<1.23.0 and >=1.16.5
seqdist 0.0.3 depends on numpy<=1.18.5
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
After installing seqdist 0.0.3 separately and upgrading numpy to 1.21.5, both training and basecalling work! Thanks for the help
Thanks for letting me know! I will add a note on the readme in case someone faces the same issue.
I get a ModuleNotFoundError when trying to run the train_original or basecall_original scripts with the required packages and demo data:
This error seems consistent with a scipy issue here: https://github.com/scipy/scipy/issues/14541 After following the instructions in this reply, the error disappears, but the command hangs on importing BasecallerImpl and BaseFast5Dataset instead.
I am using cuda 10.2, cudnn v8.7.0 with python packages and versions corresponding to the requirements.txt file. Please let me know if there is a different configuration of packages I should be using or a workaround here, thanks!