search

LooseLab / Icarust

A fully featured MinKNOW simulator for testing read until experiments.
Mozilla Public License 2.0
15 stars 5 forks source link

ReadFish unable to get sample rate #16

Closed SimiliSerpent closed 1 day ago

SimiliSerpent commented 1 month ago

Hi again,

I have an issue while trying to use Readfish, it fails to retrieve the sample rate and breaks. My command is: readfish targets --port 10000 --device Bantersaurus --experiment-name test_4_BA_PA --toml tomls/BA.toml

I get the following Readfish output and error:

2024-06-11 23:11:41,739 readfish  [abs-path-to-venv]/bin/readfish targets --port 10000 --device Bantersaurus --experiment-name test_4_BA_PA --toml tomls/BA.toml
2024-06-11 23:11:41,739 readfish chemistry=<Chemistry.SIMPLEX: 'simplex'>
2024-06-11 23:11:41,739 readfish command='targets'
2024-06-11 23:11:41,740 readfish debug_log=True
2024-06-11 23:11:41,740 readfish device='Bantersaurus'
2024-06-11 23:11:41,740 readfish dry_run=False
2024-06-11 23:11:41,740 readfish experiment_name='test_4_BA_PA'
2024-06-11 23:11:41,740 readfish host='127.0.0.1'
2024-06-11 23:11:41,740 readfish log_file=None
2024-06-11 23:11:41,740 readfish log_format='%(asctime)s %(name)s %(message)s'
2024-06-11 23:11:41,740 readfish log_level='info'
2024-06-11 23:11:41,741 readfish max_unblock_read_length_seconds=5
2024-06-11 23:11:41,741 readfish padding=0
2024-06-11 23:11:41,741 readfish port='10000'
2024-06-11 23:11:41,741 readfish throttle=0.4
2024-06-11 23:11:41,741 readfish toml='tomls/BA.toml'
2024-06-11 23:11:41,741 readfish unblock_duration=0.1
2024-06-11 23:11:41,741 readfish wait_for_ready=120
2024-06-11 23:11:41,741 readfish Version=2024.2.0
2024-06-11 23:11:41,867 readfish._read_until_client Protocol phase changed to PHASE_SEQUENCING
2024-06-11 23:11:41,868 readfish._read_until_client Protocol state changed to PROTOCOL_RUNNING
2024-06-11 23:11:41,873 readfish.targets eJzlU9tO3DAQffdXjIL62GQpi1qQeCiob22RaN/QynLscWLhG7bDsn/fcbILXdp+QSNF8tzOnDken8CPyTmRdpfw8/bbV5DBazOANhb35ykZP0BCYd8X4xB6kVEKa6tXeAUZLcpinhCciLF6g4ZrIY21U6aMMiYyMjuBnyMmBEF/2QZCHEzwGXJEabRB1VLKFyHHfQSUeTIKM5yv3lXIMiJoG7YSrYUnTMVUFrtadevxULSltiBkAUG96wQlBTvz9KGAcdGiQ09hvwOhRJyJZ0F+Yt4uJCFQr3SEWEQasMwc/hwNBvTB4VJNJj5XPAQ6HiREBToQ5OR9VWhrykj1cHe6atftKfSTfVg0r1kFc5nZEOD1Xm3ic8AShWix+8XLM5aanNthinG3YfsLvIJGecHT0oDjp/YDX69Wfcx8FLKVemiYUCphzjXXRHnZdV1xsVuAunP6GqawnwZuwwxoSStOi6Byqx+byu6zNYMnajfH1Ooi/E6t2jue8oZpz4165sZXvIQ6dwc5+YucrXPmTefaVJs8Vjt3Mw9RW9erzG0UumGeU3nlRvlrVsndLff3htv9fu82G+YFrTOhv3RumDOey3HyDxXmlDnx/Gqu2bIF9XzffL/hq9XZxcXH9qzZsExzWuRhniuXEEknibTAnnR2ky3mH7FDodY1OvneBvnwUvLG6wNp+lhdMQWJqGYXqXvkEn0gfY6ov0L0SE+IH435WvlXdcbJ8f07atjhQV1BSRP+53L9AtCxzYA=
2024-06-11 23:11:41,950 readfish.targets Configuration description:
Region anthracis (control=False).
Region applies to section of flow cell (# = applied, . = not applied):

    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................
    ############################################################............................................................

Region hum_control (control=True).
Region applies to section of flow cell (# = applied, . = not applied):

    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################
    ............................................................############################################################

2024-06-11 23:11:41,964 readfish.targets Fetching Run Configuration
Traceback (most recent call last):
  File " [abs-path-to-venv]/bin/readfish", line 8, in <module>
    sys.exit(main())
  File " [abs-path-to-venv]/lib/python3.10/site-packages/readfish/_cli_base.py", line 61, in main
    raise SystemExit(args.func(parser, args, extras))
  File "[abs-path-to-venv]/lib/python3.10/site-packages/readfish/entry_points/targets.py", line 639, in run
    worker = Analysis(
  File "[abs-path-to-venv]/lib/python3.10/site-packages/readfish/entry_points/targets.py", line 207, in __init__
    self.sample_rate = self.client.connection.device.get_sample_rate().sample_rate
  File "[abs-path-to-venv]/lib/python3.10/site-packages/minknow_api/device_service.py", line 1346, in get_sample_rate
    return run_with_retry(self._stub.get_sample_rate,
  File "[abs-path-to-venv]/lib/python3.10/site-packages/minknow_api/device_service.py", line 97, in run_with_retry
    result = MessageWrapper(method(message, timeout=timeout), unwraps=unwraps)
  File "[abs-path-to-venv]/lib/python3.10/site-packages/grpc/_channel.py", line 1181, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "[abs-path-to-venv]/lib/python3.10/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNIMPLEMENTED
        details = ""
        debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-06-11T23:11:42.772480936+02:00", grpc_status:12, grpc_message:""}"
>

My Icarust config.ini file is:

[TLS]
cert-dir = [abs-path-to-icarust]/Icarust/static/tls_certs
; cert-dir = /opt/ont/minknow/conf/rpc-certs
[PORTS]
manager = 10000
position = 10001

[SEQUENCER]
channels = 3000

My Icarust toml confirguration file is:

output_path = "[abs-path-to-output-directory]"
target_yield = 100000000000
pore_type = "R10"
nucleotide_type = "DNA"

[parameters]
sample_name = "test"
experiment_name = "test_4_BA_PA"
flowcell_name = "FAQ1234"
experiment_duration_set = 10240000
device_id = "Bantersaurus"
position = "FenceSitter"
sample_rate = 4000

[[sample]]
name = "Bacteria 1"
input_genome = "docker/squiggle_arrs/mixed_ref.fa"
mean_read_length = 3000
weight = 1

And my ReadFish toml config file is as follows:

# Basecaller configuration
[caller_settings.guppy]
config = "dna_r10.4.1_e8.2_400bps_hac.cfg"
address = "ipc:///tmp/.guppy/5555"
debug_log = "live_reads.fq"

# Aligner Configuration
[mapper_settings.mappy_rs]
fn_idx_in = "refs/Bacillus_anthracis.mmi"
debug_log = "readfish_logs/live_alignments.paf"
n_threads = 4

# Region Configuration
[[regions]]
name = "anthracis"
min_chunks = 1
max_chunks = 4
targets = ["NC_003997.3"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"
above_max_chunks = "unblock"
below_min_chunks = "proceed"

[[regions]]
name = "hum_control"
control = true
min_chunks = 1
max_chunks = 4
targets = ["NC_003997.3"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"
above_max_chunks = "unblock"
below_min_chunks = "proceed"

Running under CentOS Core. I'd be happy to share any other logs or infos! I hope you can help...

Thank you in advance Ben

Adoni5 commented 1 month ago

Hi @SimiliSerpent,

I've got a fix in the works for this already! Readfish queries a new endpoint via the minknow-api to get the sample rate, which is required by the new ont-pybasecall-client-lib for Dorado server 7.3.9. Obviously this wasn't in Icarust, which implements the bare minimum in terms of endpoints.

I've got it working, but I messed around with signal generation (variable sample rates) and made that worse, so I just need to separate those two and create separate PR's.

Should be done by tomorrow 🤞🏼 Rory

SimiliSerpent commented 1 month ago

Great news! Thank you already. Does it mean that using an older version of ReadFish might solve the issue?

Ben

Adoni5 commented 1 month ago

Readfish 2023.1.1. You would also have to use an older version of dorado-basecall-server (<7.3.9).

The issue12 branch should fix your problem, are you running natively or using docker?

SimiliSerpent commented 4 weeks ago

Actually, I am using guppy for the basecall-server. Using Readfish 2023.1.1 works like a charm. I run Readfish natively. Thank you again!

Ben

(I do not close the issue, if you want your fix to work before you close, for the latest version of Readfish)

Adoni5 commented 4 weeks ago

I shall :) I tested the branch overnight, time to see the results!

Nirmal2310 commented 2 days ago

Dear @Adoni5, any lead on this issue? I am facing the same issue when running Icarust (R10 mode) with readfish (Dorado)

SimiliSerpent commented 1 day ago

Hi,

I was finally able to work with the latest Dorado release, and... it works perfectly fine! As far as I'm concerned, we could close this issue. Thanks again @Adoni5 !!

@Nirmal2310 , are you using the issue12 branch of Icarust ? (install it with git clone --single-branch --branch issue12/add_barcoding_kits_and_sample_rate https://github.com/LooseLab/Icarust).

Ben

Adoni5 commented 1 day ago

Hey @Nirmal2310 -

I've merged the issue12 branch into main, which should fix this issue. It also implements setting the sample_rate and sequencing_speed in the simulation profile TOML. See https://github.com/LooseLab/Icarust/pull/13 and https://github.com/LooseLab/Icarust/pull/14.

If you pull down the latest changes and run, this should all work fine. The new parameters are also described in the README table. The docker image has also been brought in line with the latest version of main. I'm going to close this issue for now, but if you get a chance to test this and it doesn't work, please reopen it.

Many Thanks, Rory