LooseLab / readfish

CLI tool for flexible and fast adaptive sampling on ONT sequencers
https://looselab.github.io/readfish/
GNU General Public License v3.0
167 stars 31 forks source link

Issues running test simulation on guppy 6 minKnow 5 branch #205

Closed santiago-es closed 2 years ago

santiago-es commented 2 years ago

Hi all,

I'm fairly new to ONT sequencing and I'm attempting to use readfish for adaptive sampling.

I installed readfish using the guppy 6 and minknow branch, and it seems to have done so correctly.

When I attempt to run a simulation, I downloaded the bulk Fast5 playback file for testing readfish and changed the specified toml file as listed in the instructions, pointing the variable simulation to the fast5 playback file. I went into the MinKnow GUI and reloaded scripts and attempted to start sequencing for the simulation to begin but the sequencing run fails. I have our MinION MK1B plugged in with a configuration test cell inserted, and in the sequencing setup specified the flow cell corresponding to the edited toml file (MIN106, same as in the instructions).

Any idea where I might be going wrong? Thanks in advance for your help!

santiago-es commented 2 years ago

BTW I am doing this on Ubuntu 22.04 and this is what my MinKnow GUI looks like when attempting to run a simulation

image

it then fails at around ~10 min "due to internal error" image

santiago-es commented 2 years ago

Here is my toml file I am attempting to use:

https://pastebin.com/YBFy38UZ

santiago-es commented 2 years ago

as an update: I've tried with 2 different configuration test cells and MK1Bs, runs into the same issue. I imagine its an issue with my .toml file or the ability of the MinION to correctly point to it when setting up the run.

nick297 commented 2 years ago

Are you running readfish yet, or just the simulation when you get the error? Your toml file looked fine to me, but the authors will be better judges of that.

santiago-es commented 2 years ago

I was able to successfully install readfish in a conda environment, but i havent been able to even get the simulation to start. After restarting my computer, my minknow gui is now asking me for the host server so I think I'm making some progress.

alexomics commented 2 years ago

Ah sorry this slipped through my inbox. There’s been some changes to MinKNOW, I’m working on updating our docs.

The changes are described in https://community.nanoporetech.com/posts/simulating-a-minion-in-min (login required).

In short:

You can start the MinKNOW service with a simulation MinION by ending the MinKNOW service and starting it with a simulation parameter:

cd "C:\Program Files\OxfordNanopore\MinKNOW\bin\"
mk_manager_svc.exe --simulated-minion-devices 1

or

cd /opt/ont/minknow/bin
mk_manager_svc --simulated-minion-devices 1

There maybe some extra steps with stopping/starting the service. Please let me know if you hit any problems

santiago-es commented 2 years ago

Thanks for this alex. I'm attempting to try it but after reboot my MinKnow is stuck on a connection manager page and doesnt navigate to the gui with the MinION connection im used to so working on solving that atm

santiago-es commented 2 years ago

Was able to get it running. The mk_manager_svc --simulated-minion-devices 1 solution worked, only required running

sudo service minknow stop --> start mk manager with sim device sudo service minknow start

and then opening the gui for it to detect the simulated device. Thanks!

santiago-es commented 2 years ago

I'm back haha.

So I was able to successfully unblock all reads and moved on to testing an enrichment on the playback data.

First I started my guppy_basecall_server with:

guppy_basecall_server --config dna_r9.4.1_450bps_fast.cfg -p 5555 -l /tmp/guppy

I placed the basecall server ip and port# in my .toml file (which is the one from the tutorial / README with the appropriate edits).

I then ran: readfish targets --device MS00000 --experiment-name target2 --toml human_chr_selection.toml --log-file ru_test.log --port 9502 which seems to successfully connect (in my MinKNOW GUI, I see the "This is a live run. Unblocks may occur" message -- no others though, i.e. not the output of readfish validate) but then the throttle times are much slower than expected:

image

What might be going on here?

Here is my toml file for reference, I changed it to only select for chr1

image

I'm using the guppy6_minknow5 branch. I've also run readfish summary on an attempt at enrichment on the fast5 playback and it doesnt seem like any enrichment happened.

Something else that seems worth mentioning is that attempting to add "-x 'cuda:0' " to the end of my guppy_basecall_server start fails with a potentially depleted memory error? I thought this was odd, given I have an RTX 3080 and cuda-toolkit, nvidia drivers installed (nvidia -smi runs, nvcc runs in the conda environment cuda was installed in)

santiago-es commented 2 years ago

OK, I think I figured it out!

I think the final issue that popped up was my GPU basecalling was misconfigured and even though the GPU-Guppy was installed, the basecall server that was being created was still using the CPU-guppy

For others who may have this same issue on the guppy6_minknow5 branch here is what works from start to finish:

Follow Miles Benton's instructions for live GPU basecalling. I got this to work on Ubuntu 22 with the Ubuntu 20 repos, Guppy 6 and Minknow-core 5.1

Next install readfish following the guppy6_minknow5 branch instructions.

Before beginning a simulation, it will be necessary to stop the minknow service. On Linux you do this with:

sudo service minknow stop

Then run sudo /opt/ont/minknow/bin/mk_manager_svc --simulated-minion-devices 1

pointing to whatever the appropriate folder is for your Minknow installations bin. this starts MinKnow with a simulated device.

When it comes time to unblock, it will be necessary to reload scripts by clicking three dots on the top right of the experiment start / run-naming page of the MinKNOW GUI after editing the _sequencing.toml file for a simulated run.

After that the readfish instructions should work appropriately and you should see the gpu basecalling work appropriately.