search

nanoporetech / megalodon

Megalodon is a research command line tool to extract high accuracy modified base and sequence variant calls from raw nanopore reads by anchoring the information rich basecalling neural network output to a reference genome/transriptome.
Other
197 stars 30 forks source link

Error starting megalodon #197

Closed marcDabad closed 3 years ago

marcDabad commented 3 years ago

I am trying to run megaldon and I have an error:

megalodon input_fast5/ --guppy-params "-d ~/software/rerio/basecall_models/" --guppy-config res_dna_r941_min_modbases_5mC_CpG_v001.cfg --outputs basecalls mappings mod_mappings mods --reference index/GRCh38.primary_assembly.genome.fa --mod-motif m CG 0 --devices 1 --processes 20 --guppy-server-path ~/software/guppy/ont-guppy_5.0.14/bin/guppy_basecall_server --output-directory output_megalodon/ --write-mods-text
[17:33:37] Running Megalodon version 2.3.4
[17:33:37] Loading guppy basecalling backend
****************************************************************************************************
        ERROR: Guppy server initialization failed. See guppy logs in [--output-directory] for more details.
                Try running the guppy server initialization command found in log.txt in order to pinpoint the source of this issue.
****************************************************************************************************

At log.txt file, I have this:

[17:33:37] Running Megalodon version 2.3.4
DBG 17:33:37 : Command: """/home/marc/miniconda3/envs/megalodon/bin/megalodon input/210921_EASI_26_P5168_FAQ65185/210921_EASI_26_P5168_FAQ65185/210921_EASI_26_P5168_FAQ65185/20210921_1537_X1_FAQ65185_9ba2a3d7/fast5/ --guppy-params -d ~/software/rerio/basecall_models/ --guppy-config res_dna_r941_min_modbases_5mC_CpG_v001.cfg --outputs basecalls mappings mod_mappings mods --reference index/GRCh38.primary_assembly.genome.fa --mod-motif m CG 0 --devices 1 --processes 10 --guppy-server-path /home/marc/software/guppy/ont-guppy_5.0.14/bin/guppy_basecall_server --output-directory output_megalodon/ --write-mods-text""" --- MainProcess-MainThread megalodon.py:1750
[17:33:37] Loading guppy basecalling backend
DBG 17:33:37 : Guppy version: "5.0.14" --- MainProcess-MainThread backends.py:869
DBG 17:33:37 : Pyguppy version: "5.0.14" --- MainProcess-MainThread backends.py:870
DBG 17:33:37 : guppy server init command: "/home/marc/software/guppy/ont-guppy_5.0.14/bin/guppy_basecall_server -p auto -l output_megalodon/guppy_log -c res_dna_r941_min_modbases_5mC_CpG_v001.cfg --post_out --quiet -x cuda:1 -d ~/software/rerio/basecall_models/" --- MainProcess-MainThread backends.py:944
****************************************************************************************************
        ERROR: Guppy server initialization failed. See guppy logs in [--output-directory] for more details.
                Try running the guppy server initialization command found in log.txt in order to pinpoint the source of this issue.
****************************************************************************************************

When I try to run guppy_basecall_server I have no error. Actually, it looks is working.

guppy_basecaller_server.....log

2021-10-15 17:39:05.243736 [guppy/message] ONT Guppy basecall server software version 5.0.14+8f53ee9, client-server API version 7.0.0
log path:            output_megalodon/guppy_log
chunk size:          2000
chunks per runner:   512
max queued reads:    2000
num basecallers:     4
num socket threads:  2
max returned events: 50000
gpu device:          cuda:1
kernel path:
runners per device:  4
2021-10-15 17:39:05.243998 [guppy/info] crashpad_handler not supported on this platform.
2021-10-15 17:39:05.244613 [guppy/info] Listening on port 38485.
2021-10-15 17:39:05.384643 [guppy/info] CUDA device 1 (compute 7.5) initialised, memory limit 25388515328B (21996634112B free)
2021-10-15 17:39:07.430331 [guppy/message]
Config loaded:
config file:               /home/marc/software/rerio/basecall_models/res_dna_r941_min_modbases_5mC_CpG_v001.cfg
model file:                /home/marc/software/rerio/basecall_models/res_dna_r941_min_modbases_5mC_CpG_v001.jsn
model version id           None
adapter scaler model file: None
2021-10-15 17:39:07.431970 [guppy/message] Starting server on port: 38485

And I see it running at nvidia-smi

|    1   N/A  N/A    351097      C   ...bin/guppy_basecall_server     3687MiB

I tried it with guppy 5.0.14, 5.0.15 and 5.0.16 with the same error.

Thanks for this tool.

marcus1487 commented 3 years ago

TL;DR replace the ~ in the path with the full path to your home directory.

The issue is the ~ character in your command. The standalone guppy call on the command line performs the expansion of this character to your home directory, while within megalodon the expansion is not completed as the command is sent via a non-interactive shell within python. And the guppy basecall server does not accept paths with a tilde in them. This should resolve your issue. Best of luck in your research!

marcDabad commented 3 years ago

Oh!

It was exactly my error.

I hope this question will help future users and save their time.

Thanks for your help and job.

Fatihlrcfs commented 3 years ago

Hi all,

I wrote about the same error. even ı provide the full path, I am still getting the same error. While providing the full path, am I missing something? I am wondering where is the problematic part of my code. Thanks.

megalodon /cluster/lrcfs/2397405/projects/nanopore_testing/analysis/exercise/meteore/fast5/ --outputs mods --reference wholegenome --mod-motif m CG 0 --write-mods-text --processes 10 --guppy-server-path /cluster/lrcfs/2397405/bin/ont-guppy-cpu-5.0.16/bin/guppy_basecall_server --guppy-params "-d ~/cluster/lrcfs/2397405/bin/rerio/basecall_models/ --num_callers 10" --guppy-timeout 400 --guppy-config res_dna_r941_min_modbases_5mC_CpG_v001.cfg --overwrite [11:48:22] Running Megalodon version 2.3.4 [11:48:22] Loading guppy basecalling backend

    ERROR: Guppy server initialization failed. See guppy logs in [--output-directory] for more details.
            Try running the guppy server initialization command found in log.txt in order to pinpoint the source of this issue.

[11:48:22] Running Megalodon version 2.3.4 DBG 11:48:22 : Command: """/cluster/lrcfs/ftiras/ftiras_env/envs/nanoplot/bin/megalodon /cluster/lrcfs/2397405/projects/nanopore_testing/analysis/exercise/meteore/fast5/ --outputs mods --reference wholegenome --mod-motif m CG 0 --write-mods-text --processes 10 --guppy-server-path /cluster/lrcfs/2397405/bin/ont-guppy-cpu-5.0.16/bin/guppy_basecall_server --guppy-params -d ~/cluster/lrcfs/2397405/bin/rerio/basecall_models/ --num_callers 10 --guppy-timeout 400 --guppy-config res_dna_r941_min_modbases_5mC_CpG_v001.cfg --overwrite""" --- MainProcess-MainThread megalodon.py:1750 [11:48:22] Loading guppy basecalling backend DBG 11:48:22 : Guppy version: "5.0.16" --- MainProcess-MainThread backends.py:869 DBG 11:48:22 : Pyguppy version: "5.0.16" --- MainProcess-MainThread backends.py:870 DBG 11:48:22 : guppy server init command: "/cluster/lrcfs/2397405/bin/ont-guppy-cpu-5.0.16/bin/guppy_basecall_server -p auto -l megalodon_results/guppy_log -c res_dna_r941_min_modbases_5mC_CpG_v001.cfg --post_out --quiet -d ~/cluster/lrcfs/2397405/bin/rerio/basecall_models/ --num_callers 10" --- MainProcess-MainThread backends.py:944

ERROR: Guppy server initialization failed. See guppy logs in [--output-directory] for more details.
    Try running the guppy server initialization command found in log.txt in order to pinpoint the source of this issue.
Fatihlrcfs commented 3 years ago

Hi all,

I solved this problem by opening the Rerio data configuration (which is not a real configuration file just an address for downloading) and downloading the compressed configuration. then unzip this configuration in the data folder. I hope it fixes your problem Also. thanks

marcus1487 commented 3 years ago

See the rerio README for the recommended model download procedure using the rerio/download_model.py script. I also noticed that in your command posted indicating that you used the "full path" you included a leading ~. This is not a full path, but one that bash needs to expand. A full path should start with a leading /. I hope this helps resolve this issue.