bonito with openvino error

[studyfranco]

Hello,

I follow the instruction to install but with little modifications. (I use conda envs)

conda create --name bonito python=3.7
conda activate bonito
git clone -b openvino https://github.com/dkurt/bonito

after the pip install requirement I obtain this error message:

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.                                                                                                                                                                           
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.                                                                                                                                                                                                                                                                  
openvino-python 2021.1 requires numpy==1.16.3, but you'll have numpy 1.19.2 which is incompatible. 

when I do "bonito download --all" I obtain this output:

[downloading models]
[downloaded dna_r9.4.1.zip]                                                                         
[downloading training data]
[downloaded dna_r9.4.1.hdf5]                                                                        
[converting dna_r9.4.1.hdf5]
> preparing training chunks

1250345it [05:31, 3774.90it/s]
Traceback (most recent call last):
  File "/home/sysfate/.conda/envs/bonito/venv3/bin/bonito", line 11, in <module>
    load_entry_point('ont-bonito', 'console_scripts', 'bonito')()
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/__init__.py", line 39, in main
    args.func(args)
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/download.py", line 93, in main
    File(__models__, train).download()
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/download.py", line 68, in download
    convert(args)
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/convert.py", line 118, in main
    training_chunks = filter_chunks(training_chunks)
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/convert.py", line 87, in filter_chunks
    mu, sd = np.mean(chunks.target_lengths), np.std(chunks.target_lengths)
AttributeError: 'ChunkDataSet' object has no attribute 'target_lengths'

Have you any idea why I obtain this error ? maybe when I install openvivo, the numpy version are bad ? Or something else ?

[iiSeymour]

Hey @studyfranco

The OpenVino branch is still a work in process and I would expect a few issues until it's merged. The issue you are seeing is related to preparing the training data only so if you just want to try basecalling with the OpenVino backend you can ignore this error for now.

[studyfranco]

Hey @iiSeymour

Thank you for the rapid reply. I do what you suggest, and I try to create my basecall. bonito basecaller dna_r9.4.1 --save-ctc --use_openvino --reference possibleSeq.mmi /mnt/Data/Team_projects/fast5_pass > basecalls.sam

But I obtain this error:

> loading model
Traceback (most recent call last):
  File "/home/sysfate/.conda/envs/bonito/bin/bonito", line 33, in <module>
    sys.exit(load_entry_point('ont-bonito', 'console_scripts', 'bonito')())
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/__init__.py", line 39, in main
    args.func(args)
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/basecaller.py", line 30, in main
    use_openvino=args.use_openvino,
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/util.py", line 332, in load_model
    model = Model(toml.load(config))
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/model.py", line 81, in __init__
    self.stride = config['block'][0]['stride'][0]
KeyError: 'block'

I am a little bit lost now, what do you suggest ?

[iiSeymour]

Ah that's my fault, I've pushed a new pre-trained model in preparation for v0.3.0 next week that has a different config.

Can you delete the dna_r9.4.1 directory in bonito/models and swap the model URL here to "nas0uhf46fd1lh2jndhx2a54a9vvhxp4.zip" then run bonito download --models again.

[studyfranco]

I followed your instructions. With my next try I obtain this error:

> loading model
Traceback (most recent call last):
  File "/home/sysfate/.conda/envs/bonito/bin/bonito", line 33, in <module>
    sys.exit(load_entry_point('ont-bonito', 'console_scripts', 'bonito')())
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/__init__.py", line 39, in main
    args.func(args)
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/basecaller.py", line 30, in main
    use_openvino=args.use_openvino,
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/util.py", line 346, in load_model
    model = OpenVINOModel(model, half, dirname)
  File "/home/sysfate/.conda/envs/bonito/bonito/bonito/openvino/model.py", line 19, in __init__
    model_name = model.config['model'] + ('_fp16' if half else '')
TypeError: unsupported operand type(s) for +: 'dict' and 'str'

inside model.config['model'] you have {'package': 'bonito.ctc'} What are your suggestion ? Thank you lot of for your help !

[dkurt]

Hi!

openvino-python 2021.1 requires numpy==1.16.3, but you'll have numpy 1.19.2 which is incompatible.

Please try an updated version of the branch. I modified requirements.txt so it installs numpy==1.16.3.