epi2me-labs / wf-basecalling

Other
34 stars 14 forks source link

[error]: you have entered an unknown model name. #2

Closed neurogenetics1 closed 1 year ago

neurogenetics1 commented 1 year ago

What happened?

Hi,

I am getting an error when using a workflow EPI2ME in nextflow. I used the command as it is described on the GitHub page and –the help tutorial. It requires an input file and a model file. However, the dorado tool cannot find any of the available models, although I am using one of the suggested ones. There might be a configuration error. I have been trying to solve this problem for more than a week and have decided to get help. I will be grateful if you can solve this error.

Operating System

ubuntu 20.04

Workflow Execution

Command line

Workflow Execution - EPI2ME Labs Versions

nextflow/22.04.5

Workflow Execution - CLI Execution Profile

Singularity

Workflow Version

wf-basecalling v0.2.0-g2d733e8

Relevant log output

Here is my command:

module load nextflow/22.04.5
module load singularity/3.10.5

cd AZ140021
export SINGULARITY_CACHEDIR=/data/nanopore/output

nextflow run -profile biowulf epi2me-labs/wf-basecalling  --basecaller_cfg “[dna_r10.4.1_e8.2_400bps_hac@v4.0.0](mailto:dna_r10.4.1_e8.2_400bps_hac@v4.0.0)” --input /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/fast5_pass/ --ref /data/nanopore/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa --dorado_ext fast5 --out_dir /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/output2023 --basecaller_basemod_threads 4

Output and error:

Core Nextflow options
 revision                  : master
 runName                   : voluminous_jones
 containerEngine           : singularity
 launchDir                 : /vf/users/nanopore/AZ140021
 workDir                   : /vf/users/nanopore/AZ140021/work
 projectDir                : /home/akcimenf2/.nextflow/22.04.5/assets/epi2me-labs/wf-basecalling
 userName                  : akcimenf2
 profile                   : biowulf
 configFiles               : /home/akcimenf2/.nextflow/22.04.5/assets/epi2me-labs/wf-basecalling/nextflow.config, /vf/users/nanopore/AZ140021/nextflow.config

Input Options
 input                     : /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/fast5_pass/
 ref                       : /data/nanopore/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa

Output Options
 out_dir                   : /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/output2023

Basecalling options
 basecaller_cfg            : [dna_r10.4.1_e8.2_400bps_hac@v4.0.0](mailto:dna_r10.4.1_e8.2_400bps_hac@v4.0.0)
 basecaller_basemod_threads: 4

Error executing process > 'wf_dorado:dorado (1)'

Caused by:
  Process `wf_dorado:dorado (1)` terminated with an error exit status (1)

Command executed:

  echo '***'
  echo 'Available models:'
  list-models | sed 's,^,- ,' | sed "s,${DRD_MODELS_PATH}/,,"
  echo '***'
  echo 'You selected:'
  echo "Basecalling model: [dna_r10.4.1_e8.2_400bps_hac@v4.0.0](mailto:dna_r10.4.1_e8.2_400bps_hac@v4.0.0)"
  echo "Remora model     : null"
  echo '***'
  echo 'A file open error below indicates that you have entered an unknown model name.'
  echo 'It is possible the model you selected worked previously but has been updated to a new version.'
  echo 'Resubmit this workflow with an appropriate model from the model list above.'
  echo '***'

  dorado basecaller         [${DRD_MODELS_PATH}/dna_r10.4.1_e8.2_400bps_hac@v4.0.0](mailto:$%7bDRD_MODELS_PATH%7d/dna_r10.4.1_e8.2_400bps_hac@v4.0.0) .                           --device cuda:all | samtools view -b -o 0.ubam -

Command exit status:
  1

Command output:
  ***
  Available models:
  - [dna_r10.4.1_e8.2_260bps_fast@v4.0.0](mailto:dna_r10.4.1_e8.2_260bps_fast@v4.0.0)
  - [dna_r10.4.1_e8.2_260bps_hac@v4.0.0](mailto:dna_r10.4.1_e8.2_260bps_hac@v4.0.0)
  - [dna_r10.4.1_e8.2_260bps_sup@v4.0.0](mailto:dna_r10.4.1_e8.2_260bps_sup@v4.0.0)
  - [dna_r10.4.1_e8.2_400bps_fast@v4.0.0](mailto:dna_r10.4.1_e8.2_400bps_fast@v4.0.0)
  - [dna_r10.4.1_e8.2_400bps_fast@v4.0.0_5mCG_5hmCG@v2](mailto:dna_r10.4.1_e8.2_400bps_fast@v4.0.0_5mCG_5hmCG@v2)
  - [dna_r10.4.1_e8.2_400bps_hac@v4.0.0](mailto:dna_r10.4.1_e8.2_400bps_hac@v4.0.0)
  - [dna_r10.4.1_e8.2_400bps_hac@v4.0.0_5mCG_5hmCG@v2](mailto:dna_r10.4.1_e8.2_400bps_hac@v4.0.0_5mCG_5hmCG@v2)
  - [dna_r10.4.1_e8.2_400bps_sup@v4.0.0](mailto:dna_r10.4.1_e8.2_400bps_sup@v4.0.0)
  - [dna_r10.4.1_e8.2_400bps_sup@v4.0.0_5mCG_5hmCG@v2](mailto:dna_r10.4.1_e8.2_400bps_sup@v4.0.0_5mCG_5hmCG@v2)
  - [dna_r10.4.2_e8.2_4khz_stereo@v1.0](mailto:dna_r10.4.2_e8.2_4khz_stereo@v1.0)
  - [dna_r9.4.1_e8_fast@v3.4](mailto:dna_r9.4.1_e8_fast@v3.4)
  - [dna_r9.4.1_e8_fast@v3.4_5mCG@v0](mailto:dna_r9.4.1_e8_fast@v3.4_5mCG@v0)
  - [dna_r9.4.1_e8_hac@v3.3](mailto:dna_r9.4.1_e8_hac@v3.3)
  - [dna_r9.4.1_e8_hac@v3.4_5mCG@v0](mailto:dna_r9.4.1_e8_hac@v3.4_5mCG@v0)
  - [dna_r9.4.1_e8_sup@v3.3](mailto:dna_r9.4.1_e8_sup@v3.3)
  - [dna_r9.4.1_e8_sup@v3.4_5mCG@v0](mailto:dna_r9.4.1_e8_sup@v3.4_5mCG@v0)
  - rna003_120bps_sup@v3
  ***
  You selected:
  Basecalling model: [dna_r10.4.1_e8.2_400bps_hac@v4.0.0](mailto:dna_r10.4.1_e8.2_400bps_hac@v4.0.0)
  Remora model     : null
  ***
  A file open error below indicates that you have entered an unknown model name.
  It is possible the model you selected worked previously but has been updated to a new version.
  Resubmit this workflow with an appropriate model from the model list above.
  ***

Command error:
  [2023-01-26 10:58:18.446] [info] > Creating basecall pipeline
  [2023-01-26 10:58:18.565] [error] CUDA device requested but no devices found.
  [main_samview] fail to read the header from "-".

Work dir:
  /vf/users/nanopore/AZ140021/work/25/d1364160039a75f878052065d16636

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
SamStudio8 commented 1 year ago

Your error log contains the following message:

[2023-01-26 10:58:18.565] [error] CUDA device requested but no devices found.

This indicates Dorado cannot find a GPU which is required for basecalling. Does your computer have a GPU?

On Sat, 28 Jan 2023, 15:09 neurogenetics1, @.***> wrote:

What happened?

Hi,

I am getting an error when using a workflow EPI2ME in nextflow. I used the command as it is described on the GitHub page and –the help tutorial. It requires an input file and a model file. However, the dorado tool cannot find any of the available models, although I am using one of the suggested ones. There might be a configuration error. I have been trying to solve this problem for more than a week and have decided to get help. I will be grateful if you can solve this error. Operating System

ubuntu 20.04 Workflow Execution

Command line Workflow Execution - EPI2ME Labs Versions

nextflow/22.04.5 Workflow Execution - CLI Execution Profile

Singularity Workflow Version

wf-basecalling v0.2.0-g2d733e8 Relevant log output

Here is my command:

module load nextflow/22.04.5

module load singularity/3.10.5

cd AZ140021 export SINGULARITY_CACHEDIR=/data/nanopore/output

nextflow run -profile biowulf epi2me-labs/wf-basecalling --basecaller_cfg @.**@.)” --input /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/fast5_pass/ --ref /data/nanopore/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa --dorado_ext fast5 --out_dir /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/output2023 --basecaller_basemod_threads 4

Output and error:

Core Nextflow options

revision : master

runName : voluminous_jones

containerEngine : singularity

launchDir : /vf/users/nanopore/AZ140021

workDir : /vf/users/nanopore/AZ140021/work

projectDir : /home/akcimenf2/.nextflow/22.04.5/assets/epi2me-labs/wf-basecalling

userName : akcimenf2

profile : biowulf

configFiles : /home/akcimenf2/.nextflow/22.04.5/assets/epi2me-labs/wf-basecalling/nextflow.config, /vf/users/nanopore/AZ140021/nextflow.config

Input Options

input : /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/fast5_pass/

ref : /data/nanopore/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa

Output Options

out_dir : /data/nanopore/AZ140021/20220307_1824_2D_PAI72310_c23d05c2/output2023

Basecalling options

basecaller_cfg : @.**@.)

basecaller_basemod_threads: 4

Error executing process > 'wf_dorado:dorado (1)'

Caused by:

Process wf_dorado:dorado (1) terminated with an error exit status (1)

Command executed:

echo '***'

echo 'Available models:'

list-models | sed 's,^,- ,' | sed "s,${DRD_MODELS_PATH}/,,"

echo '***'

echo 'You selected:'

echo "Basecalling model: @.**@.)"

echo "Remora model : null"

echo '***'

echo 'A file open error below indicates that you have entered an unknown model name.'

echo 'It is possible the model you selected worked previously but has been updated to a new version.'

echo 'Resubmit this workflow with an appropriate model from the model list above.'

echo '***'

dorado basecaller @.**@.) . --device cuda:all | samtools view -b -o 0.ubam -

Command exit status:

1

Command output:


Available models:

  • @.**@.)

  • @.**@.)

  • @.**@.)

  • @.**@.)

  • @.**@*.**@*.**@.)

  • @.**@.)

  • @.**@*.**@*.**@.)

  • @.**@.)

  • @.**@*.**@*.**@.)

  • @.**@.)

  • @.**@.)

  • @.**@*.**@*.**@.)

  • @.**@.)

  • @.**@*.**@*.**@.)

  • @.**@.)

  • @.**@*.**@*.**@.)

  • @.***


    You selected:

    Basecalling model: @.**@.)

    Remora model : null


    A file open error below indicates that you have entered an unknown model name.

    It is possible the model you selected worked previously but has been updated to a new version.

    Resubmit this workflow with an appropriate model from the model list above.


Command error:

[2023-01-26 10:58:18.446] [info] > Creating basecall pipeline

[2023-01-26 10:58:18.565] [error] CUDA device requested but no devices found.

[main_samview] fail to read the header from "-".

Work dir:

/vf/users/nanopore/AZ140021/work/25/d1364160039a75f878052065d16636

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-basecalling/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIN6OWIM6EWFSAO25G6HV3WUUZA5ANCNFSM6AAAAAAUJWDKUU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

neurogenetics1 commented 1 year ago

Yes, but we had not specified it in the configuration file we used. It works now. Thank you.

SamStudio8 commented 1 year ago

Thanks for letting me know, @neurogenetics1! Out of interest were you missing a clusterOption or containerOption (or something else)?

On Mon, 30 Jan 2023, 20:08 neurogenetics1, @.***> wrote:

Closed #2 https://github.com/epi2me-labs/wf-basecalling/issues/2 as completed.

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-basecalling/issues/2#event-8392067840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIN6OWDJ44EY5O3LYWFVLLWVANUPANCNFSM6AAAAAAUJWDKUU . You are receiving this because you commented.Message ID: @.***>

neurogenetics1 commented 1 year ago

We had specified “clusterOptions = ' --gres=lscratch:400'”.

The below lines were missing in our configuration file in our system.

        withLabel:gpu {
            queue = 'gpu'
            clusterOptions = ' --gres=gpu:v100x:1,lscratch:400'
            containerOptions = "--nv"
        }
        withName:make_mmi {
            memory = '16 G'
        }
        withName:getVersions {
            executor = 'local'
        }
        withName:getParams {
            executor = 'local'
        }
    }
neurogenetics1 commented 1 year ago

We had specified “clusterOptions = ' --gres=lscratch:400'”.

The below lines were missing in our configuration file in our system.

        withLabel:gpu {
            queue = 'gpu'
            clusterOptions = ' --gres=gpu:v100x:1,lscratch:400'
            containerOptions = "--nv"
        }
        withName:make_mmi {
            memory = '16 G'
        }
        withName:getVersions {
            executor = 'local'
        }
        withName:getParams {
            executor = 'local'
        }
    }

Thanks again! From: Sam Nicholls @.> Reply-To: epi2me-labs/wf-basecalling @.> Date: Tuesday, January 31, 2023 at 3:25 AM To: epi2me-labs/wf-basecalling @.> Cc: "Akcimen, Fulya (NIH/NIA/IRP) [F]" @.>, Mention @.***> Subject: [EXTERNAL] Re: [epi2me-labs/wf-basecalling] [error]: you have entered an unknown model name. (Issue #2)

Thanks for letting me know, @neurogenetics1! Out of interest were you missing a clusterOption or containerOption (or something else)?

On Mon, 30 Jan 2023, 20:08 neurogenetics1, @.***> wrote:

Closed #2 https://github.com/epi2me-labs/wf-basecalling/issues/2 as completed.

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-basecalling/issues/2#event-8392067840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIN6OWDJ44EY5O3LYWFVLLWVANUPANCNFSM6AAAAAAUJWDKUU . You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fepi2me-labs%2Fwf-basecalling%2Fissues%2F2%23issuecomment-1409951261&data=05%7C01%7Cfulya.akcimen%40nih.gov%7C6d7b387c1ef845bb0fde08db0364b915%7C14b77578977342d58507251ca2dc2b06%7C0%7C0%7C638107503385806820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aDQwYGGFYNWmpRpzm9a5N74bi3YAbLcCpH5QJxqQ3Lw%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FA5PC6KERZOONLQQLTEL2KMDWVDD7ZANCNFSM6AAAAAAUJWDKUU&data=05%7C01%7Cfulya.akcimen%40nih.gov%7C6d7b387c1ef845bb0fde08db0364b915%7C14b77578977342d58507251ca2dc2b06%7C0%7C0%7C638107503385806820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9LlEaRiwcJiKsH8EX8vz%2FWruM9PM2t1E05P9hY%2F9j1g%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.