Connection error - Githubissues

ulloaa commented 1 year ago

Hello,

We are getting an error in linux using your test example (using guppy basecaller). The test example using basecalled data works fine but when doing the second example:

run -f ref/ECOLI.fa -o PRJEB22772 -i PRJEB22772/* -t4 \ --host /guppy/4.2.2/bin/guppy_basecall_server

we get the following error:

ConnectionError: Connect with 'dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac' failed: [bad_reply] Could not interpret message from server for request: LOAD_CONFIG. Reply: INVALID_PROTOCOL [2022-11-10 05:37:06.553666] [0x00002aaab5e8e700] [info] Connection error. [bad_reply] Could not interpret message from server for request: LOAD_CONFIG. Reply: INVALID_PROTOCOL

Could you please provide some guidance on this error?

Thanks!

lpryszcz commented 1 year ago

Hi, most likely you have wrong version of pyguppyclient installed. I guess with guppy v4.2.2 you'll need pyguppyclient v0.0.7a1. More info in docs https://modphred.readthedocs.io/en/latest/install.html#which-pyguppyclient-version-should-i-install

ulloaa commented 1 year ago

Thanks for the advice. After downgrading pyguppyclient to v0.0.7a1 we get a slightly different error:

ConnectionError: Connect with 'dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac' failed: Invalid argument

I am not sure how to interpret the error. Do you have any advice? Thanks!

lpryszcz commented 1 year ago

That's odd error. Could you please send the full command you're using?

ulloaa commented 1 year ago

Sure. First I run the basecaller server:

guppy_basecall_server --config dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac.cfg -p 5555 -l /tmp/subdir -x 'cuda:0'

Then I downloaded the test dataset:

wget https://public-docs.crg.es/enovoa/public/lpryszcz/src/modPhred/test/ -q -r -c -nc -np -nH --cut-dirs=6 --reject="index.html*"

Then I ran modphred:

run -f ref/ECOLI.fa -o PRJEB22772 -i PRJEB22772/* -t4 --host guppy_basecall_server -p 5555

and the full error was:

[2022-11-14 13:27:57] ===== Welcome, welcome to modPhred pipeline! ===== [2022-11-14 13:27:57] Output directory exits. Steps completed previously will be skipped! [mem: 115 MB] [2022-11-14 13:27:57] Encoding modification info from 3 directories... [mem: 115 MB] [2022-11-14 13:27:57] PRJEB22772/MARC_ZFscreens_R9.4_1D-Ecoli-run_FAF05145 with 4 Fast5 file(s)... [mem: 115 MB] multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/modphred/conda/envs/1.0b/lib/python3.7/multiprocessing/pool.py", line 121, in worker result = (True, func(*args, **kwds)) File "/modphred/1.0b/src/guppy_encode_live.py", line 109, in basecalling_worker client.connect() File "/modphred/conda/envs/1.0b/lib/python3.7/site-packages/pyguppyclient/client.py", line 81, in connect self.pcl_client.get_error_message()) ConnectionError: Connect with 'dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac' failed: Invalid argument """ `The above exception was the direct cause of the following exception:` Traceback (most recent call last): File "/modphred/1.0b/bin/run", line 503, in <module> main() File "/modphred/1.0b/bin/run", line 433, in main o.MaxModsPerBase, o.recursive) File "/modphred/1.0b/src/guppy_encode_live.py", line 226, in mod_encode for ii, (fn, ofn, reads) in enumerate(p.imap(basecalling_worker, args), 1): File "/modphred/conda/envs/1.0b/lib/python3.7/multiprocessing/pool.py", line 748, in next raise value ConnectionError: Connect with 'dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac' failed: Invalid argument

lpryszcz commented 1 year ago

ok, so you either have to (see usage for details)

start guppy_basecall_serverand provide --host IP --port PORT, where IP is the IP of the host running guppy (by default localhost) and PORT is the guppy port (by default 5555) - this is useful if you have remote guppy server already running
OR provide guppy binaries as --host and modPhred will execute guppy internally - that's recommended

Note, either way, you'll need to provide --config as well.

In your case, you start the guppy_basecall_server and ask modPhred to start another instance. That unnecessary.

Something like this should work for you:

run -f ref/ECOLI.fa -o PRJEB22772 -i PRJEB22772/* -t4 \
  --host ~/src/ont-guppy_v4xxx/bin/guppy_basecall_server \
  --config dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac.cfg

ulloaa commented 1 year ago

Unfortunately I keep running into the same error, so I tried the step by step approach (without running the basecall server separately):

guppy_encode_live.py -i PRJEB22772/* -o OUTPUT \
    -c dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac.cfg \
    --host guppy_basecall_server

and I get the same error as before:

[2022-11-15 12:52:47] Encoding modification info from 2 directories... [mem:    44 MB]
[2022-11-15 12:52:47]  PRJEB22772/MARC_ZFscreens_R9.4_1D-Ecoli-run_FAF05145 with 4 Fast5 file(s)... [mem:    44 MB]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/modphred/conda/envs/1.0b/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/modphred/1.0b/src/guppy_encode_live.py", line 109, in basecalling_worker
    client.connect()
  File "/modphred/conda/envs/1.0b/lib/python3.7/site-packages/pyguppyclient/client.py", line 81, in connect
    self.pcl_client.get_error_message())
ConnectionError: Connect with 'dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac' failed: Invalid argument
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/modphred/1.0b/src/guppy_encode_live.py", line 329, in <module>
    main()
  File "/modphred/1.0b/src/guppy_encode_live.py", line 324, in main
    o.recursive)
  File "/modphred/1.0b/src/guppy_encode_live.py", line 226, in mod_encode
    for ii, (fn, ofn, reads) in enumerate(p.imap(basecalling_worker, args), 1):
  File "/modphred/conda/envs/1.0b/lib/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
ConnectionError: Connect with 'dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac' failed: Invalid argument

ulloaa commented 1 year ago

The error seems to be coming from pyguppyclient:

https://github.com/nanoporetech/pyguppyclient/issues/5

... and this happens even before the basecall server is started.

ulloaa commented 1 year ago

Let me try downgrading pyguppyclient to 0.0.6 and running with with lower versions of guppy, just to see if this is a compatibility problem.

ulloaa commented 1 year ago

OK, I should have known. Using pyguppyclient 0.0.6 and guppy 3.4.3 I get the "bad request" error shown in https://github.com/nanoporetech/pyguppyclient/issues/5.

lpryszcz commented 1 year ago

Hi, have you managed to solve it?

Does guppy_basecall_server point to guppy_basecall_server binary? Usually, you need to provide full path ie. /home/some_user/src/ont-guppy.../bin/guppy_basecall_server.

guppy_encode_live.py -i PRJEB22772/* -o OUTPUT \
    -c dna_r9.4.1_450bps_modbases_dam-dcm-cpg_hac.cfg \
    --host guppy_basecall_server

ulloaa commented 1 year ago

Thank you for your advice. Providing the full path to guppy_basecall_server fixed the ConnectionError

novoalab / modPhred

Connection error #9