Runtime ModuleNotFoundError for training and basecalling

[shishir-reddy]

I get a ModuleNotFoundError when trying to run the train_original or basecall_original scripts with the required packages and demo data:

(venv3) s@bwt:~/mobile_workspace/original/basecalling_architectures$ python ./scripts/basecall_original.py \
--model bonito \
--fast5-dir ./demo_data/fast5 \
--checkpoint ./demo_data/properly_trained_bonito/bonito_checkpoint.pt \
--output-file ./demo_data/demo_basecalls.fastq
Traceback (most recent call last):
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/./scripts/basecall_original.py", line 9, in <module>
    from classes import BasecallerImpl, BaseFast5Dataset
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/src/classes.py", line 15, in <module>
    from normalization import normalize_signal_from_read_data, med_mad
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/src/normalization.py", line 5, in <module>
    from scipy.signal import find_peaks
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/venv3/lib/python3.9/site-packages/scipy/signal/__init__.py", line 310, in <module>
    from ._peak_finding import *
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/venv3/lib/python3.9/site-packages/scipy/signal/_peak_finding.py", line 8, in <module>
    from scipy.stats import scoreatpercentile
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/venv3/lib/python3.9/site-packages/scipy/stats/__init__.py", line 441, in <module>
    from .stats import *
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/venv3/lib/python3.9/site-packages/scipy/stats/stats.py", line 43, in <module>
    from . import distributions
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/venv3/lib/python3.9/site-packages/scipy/stats/distributions.py", line 11, in <module>
    from . import _discrete_distns
  File "/home/shishir/mobile_workspace/original/basecalling_architectures/venv3/lib/python3.9/site-packages/scipy/stats/_discrete_distns.py", line 19, in <module>
    from .biasedurn import (_PyFishersNCHypergeometric,
  File "biasedurn.pyx", line 1, in init scipy.stats.biasedurn
ModuleNotFoundError: No module named 'numpy.random.bit_generator'

This error seems consistent with a scipy issue here: https://github.com/scipy/scipy/issues/14541 After following the instructions in this reply, the error disappears, but the command hangs on importing BasecallerImpl and BaseFast5Dataset instead.

I am using cuda 10.2, cudnn v8.7.0 with python packages and versions corresponding to the requirements.txt file. Please let me know if there is a different configuration of packages I should be using or a workaround here, thanks!

[marcpaga]

I have added the numpy version I used on the requirements (it was missing for some reason), it is numpy==1.21.5. I also used python3.7, as I see you use 3.9. My suggestion would be that you create a new environment on python3.7 and re-install the requirements with the new numpy version constraint.

[shishir-reddy]

I have created a new environment with python3.7 and the updated requirements.txt file, but I am still seeing some dependency conflicts. It looks like there is a conflict where seqdist 0.0.3 depends on numpy <= 1.18.5:

(venv3) s@bwt:~/mobile_workspace/original/basecalling_architectures$ pip install -r requirements.txt
Collecting numpy==1.21.5 (from -r requirements.txt (line 1))
  Using cached numpy-1.21.5-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Collecting cupy-cuda102==9.4.0 (from -r requirements.txt (line 2))
  Using cached cupy_cuda102-9.4.0-cp37-cp37m-manylinux1_x86_64.whl (60.7 MB)
Collecting fast_ctc_decode==0.3.0 (from -r requirements.txt (line 3))
  Using cached fast_ctc_decode-0.3.0-cp37-cp37m-manylinux2010_x86_64.whl (508 kB)
Collecting h5py==3.7.0 (from -r requirements.txt (line 4))
  Using cached h5py-3.7.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.1 MB)
Collecting koi-cuda102==0.0.5 (from -r requirements.txt (line 5))
  Using cached koi_cuda102-0.0.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (350 kB)
Collecting ont-fast5-api==4.0.2 (from -r requirements.txt (line 6))
  Using cached ont_fast5_api-4.0.2-py3-none-any.whl (2.0 MB)
Collecting pandas==1.3.5 (from -r requirements.txt (line 7))
  Using cached pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
Collecting parasail==1.2.4 (from -r requirements.txt (line 8))
  Using cached parasail-1.2.4-py2.py3-none-manylinux2010_x86_64.whl (14.1 MB)
Collecting psutil==5.8.0 (from -r requirements.txt (line 9))
  Using cached psutil-5.8.0-cp37-cp37m-manylinux2010_x86_64.whl (296 kB)
Collecting scipy==1.7.1 (from -r requirements.txt (line 10))
  Using cached scipy-1.7.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.5 MB)
Collecting seqdist==0.0.3 (from -r requirements.txt (line 11))
  Using cached seqdist-0.0.3-py3-none-any.whl (21 kB)
Collecting torch==1.9.0 (from -r requirements.txt (line 12))
  Using cached torch-1.9.0-cp37-cp37m-manylinux1_x86_64.whl (831.4 MB)
Collecting torchaudio==0.9.0 (from -r requirements.txt (line 13))
  Using cached torchaudio-0.9.0-cp37-cp37m-manylinux1_x86_64.whl (1.9 MB)
Collecting torchvision==0.10.0 (from -r requirements.txt (line 14))
  Using cached torchvision-0.10.0-cp37-cp37m-manylinux1_x86_64.whl (22.1 MB)
Collecting tqdm==4.62.0 (from -r requirements.txt (line 15))
  Using cached tqdm-4.62.0-py2.py3-none-any.whl (76 kB)
Collecting fastrlock>=0.5 (from cupy-cuda102==9.4.0->-r requirements.txt (line 2))
  Using cached fastrlock-0.8.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_24_x86_64.whl (46 kB)
Collecting cffi (from koi-cuda102==0.0.5->-r requirements.txt (line 5))
  Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting packaging (from ont-fast5-api==4.0.2->-r requirements.txt (line 6))
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting progressbar33>=2.3.1 (from ont-fast5-api==4.0.2->-r requirements.txt (line 6))
  Using cached progressbar33-2.4.tar.gz (10 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting python-dateutil>=2.7.3 (from pandas==1.3.5->-r requirements.txt (line 7))
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz>=2017.3 (from pandas==1.3.5->-r requirements.txt (line 7))
  Using cached pytz-2023.3-py2.py3-none-any.whl (502 kB)
INFO: pip is looking at multiple versions of seqdist to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 10), -r requirements.txt (line 11), -r requirements.txt (line 2), -r requirements.txt (line 4), -r requirements.txt (line 6), -r requirements.txt (line 7), -r requirements.txt (line 8) and numpy==1.21.5 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested numpy==1.21.5
    cupy-cuda102 9.4.0 depends on numpy<1.24 and >=1.17
    h5py 3.7.0 depends on numpy>=1.14.5
    ont-fast5-api 4.0.2 depends on numpy>=1.16
    pandas 1.3.5 depends on numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10"
    parasail 1.2.4 depends on numpy
    scipy 1.7.1 depends on numpy<1.23.0 and >=1.16.5
    seqdist 0.0.3 depends on numpy<=1.18.5

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

[shishir-reddy]

After installing seqdist 0.0.3 separately and upgrading numpy to 1.21.5, both training and basecalling work! Thanks for the help

[marcpaga]

Thanks for letting me know! I will add a note on the readme in case someone faces the same issue.