bigsi search failed when using docker

[YulongNiu]

Hi,

I tried to follow the manual at https://bigsi.readme.io/docs/your-first-bigsi and run BIGSI in docker image (phelimb/bigsi:63768c2).

I got the error messages in the search step:

❯ docker run -v $PWD/example-data:/data phelimb/bigsi:63768c2 bigsi search --config /data/configs/berkeleydb.yaml CGGCGAGGAAGCGTTAAATCTCTTTCTGACG
Traceback (most recent call last):
  File "/usr/local/bin/bigsi", line 11, in <module>
    load_entry_point('bigsi==0.3.2', 'console_scripts', 'bigsi')()
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.2-py3.7.egg/bigsi/__main__.py", line 178, in main
  File "hug/api.py", line 390, in hug.api.CLIInterfaceAPI.__call__
  File "hug/interface.py", line 551, in hug.interface.CLI.__call__
  File "hug/interface.py", line 547, in hug.interface.CLI.__call__
  File "hug/interface.py", line 100, in hug.interface.Interfaces.__call__
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.2-py3.7.egg/bigsi/__main__.py", line 158, in search
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.2-py3.7.egg/bigsi/graph/bigsi.py", line 133, in __init__
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.2-py3.7.egg/bigsi/graph/index.py", line 23, in __init__
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.2-py3.7.egg/bigsi/matrix/bitmatrix.py", line 16, in __init__
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.2-py3.7.egg/bigsi/storage/base.py", line 67, in get_integer
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.2-py3.7.egg/bigsi/storage/base.py", line 21, in __getitem__
  File "/usr/local/lib/python3.7/dist-packages/bsddb3/__init__.py", line 239, in __getitem__
    return _DeadlockWrap(lambda: self.db[key])  # self.db[key]
  File "/usr/local/lib/python3.7/dist-packages/bsddb3/dbutils.py", line 67, in DeadlockWrap
    return function(*_args, **_kwargs)
  File "/usr/local/lib/python3.7/dist-packages/bsddb3/__init__.py", line 239, in <lambda>
    return _DeadlockWrap(lambda: self.db[key])  # self.db[key]
KeyError: b'number_of_rows:int'

I also pasted the outputs of build:

❯ docker run -v $PWD/example-data:/data phelimb/bigsi:63768c2 bigsi build --config /data/configs/berkeleydb.yaml /data/test1.bloom /data/test2.bloom -s s1 -s s2
INFO:bigsi.cmds.build:Building index: 0/1
DEBUG:bigsi.cmds.build:Loading /data/test1.bloom/test1.bloom
DEBUG:bigsi.cmds.build:Loading /data/test2.bloom/test2.bloom
DEBUG:bigsi.graph.bigsi:Insert sample metadata
DEBUG:bigsi.graph.bigsi:Create signature index
DEBUG:bigsi.graph.index:Transpose bitarrays
DEBUG:bigsi.graph.index:Insert rows
DEBUG:bigsi.storage.base:set bitarrays

I also tried the latest docker image phelimb/bigsi:310ef4c, it failed at build step:

❯ docker run -v $PWD/example-data:/data phelimb/bigsi:310ef4c bigsi build --config /data/configs/berkeleydb.yaml /data/test1.bloom /data/test2.bloom -s s1 -s s2

Traceback (most recent call last):
  File "/usr/local/bin/bigsi", line 11, in <module>
    load_entry_point('bigsi==0.3.8', 'console_scripts', 'bigsi')()
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.8-py3.7.egg/bigsi/__main__.py", line 324, in main
  File "/usr/local/lib/python3.7/dist-packages/hug/api.py", line 439, in __call__
    result = self.commands.get(command)()
  File "/usr/local/lib/python3.7/dist-packages/hug/interface.py", line 631, in __call__
    raise exception
  File "/usr/local/lib/python3.7/dist-packages/hug/interface.py", line 627, in __call__
    result = self.output(self.interface(**pass_to_function), context)
  File "/usr/local/lib/python3.7/dist-packages/hug/interface.py", line 123, in __call__
    return __hug_internal_self._function(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/bigsi-0.3.8-py3.7.egg/bigsi/__main__.py", line 157, in build
AssertionError

Could you please provide any suggestions?

[iqbal-lab]

Could you take a look please @Zhicheng-Liu or @leoisl ?

[Zhicheng-Liu]

Ok, I will take a look. @YulongNiu , @iqbal-lab and @leoisl

[Zhicheng-Liu]

Hi @YulongNiu , many thanks for bringing this issue to our attention. The instructions you were following and the docker images were outdated. I have fixed the README file and the Wiki pages of this repo.

Please follow our Wiki page for testing out your first bigsi: https://github.com/iqbal-lab-org/BIGSI/wiki/Constructing-a-BIGSI. Specifically if you use docker these are the commands for testing with the example data:

docker run -v $PWD:/data iqballab/bigsi:latest bigsi bloom --config example-data/configs/berkeleydb.yaml example-data/test1.ctx /data/test1.bloom
docker run -v $PWD:/data iqballab/bigsi:latest bigsi bloom --config example-data/configs/berkeleydb.yaml example-data/test2.ctx /data/test2.bloom
docker run -v $PWD:/data iqballab/bigsi:latest bigsi build --config example-data/configs/berkeleydb.yaml -b /data/test1.bloom -b /data/test2.bloom -s s1 -s s2
docker run -v $PWD:/data iqballab/bigsi:latest bigsi search --config example-data/configs/berkeleydb.yaml CGGCGAGGAAGCGTTAAATCTCTTTCTGACG

[YulongNiu]

Thank you all, it works now.