search

epi2me-labs / wf-human-variation

Other
86 stars 41 forks source link

FATAL: Relocation not allowed with starter-suid #161

Closed blanleung closed 3 months ago

blanleung commented 3 months ago

Operating System

CentOS 7

Other Linux

No response

Workflow Version

v2.0.0-g52e3698

Workflow Execution

Command line

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-human-variation \ --bam '/data10/users/bob/epi2me_test_run/data_raw/wf-human-variation-demo/demo.bam' \ --basecaller_cfg 'dna_r10.4.1_e8.2_400bps_hac_prom' \ --ref '/data10/users/bob/epi2me_test_run/data_raw/wf-human-variation-demo/demo.fasta' \ --sample_name 'DEMO' \ --snp \ -profile singularity

Workflow Execution - CLI Execution Profile

singularity

What happened?

Hi,

I'm trying to run the wf-human-variation workflow with provided demo data and singularity profile on our HPC. However it always end up with the following a fatal error: FATAL: Relocation not allowed with starter-suid. Please note I got this error with either cram_cache or lookup_clair3_model processes. Any help would be appreciated.

Thanks :)

Relevant log output

Nextflow 23.10.1 is available - Please consider updating your version to it
N E X T F L O W  ~  version 23.10.0
Launching `https://github.com/epi2me-labs/wf-human-variation` [stupefied_jang] DSL2 - revision: 52e3698431 [master]

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-human-variation v2.0.0-g52e3698
--------------------------------------------------------------------------------
Core Nextflow options
  revision       : master
  runName        : stupefied_jang
  containerEngine: singularity
  container      : [withLabel:wf_human_snp:ontresearch/wf-human-variation-snp:shaca21b1351b9465f98ecdb10e01bd48dbf05e0e63, withLabel:wf_human_sv:ontresearch/wf-human-variation-sv:sha69e7a43c6f21c0ea2083e9719257fb44bc35c2ba, withLabel:wf_human_mod:ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb, withLabel:wf_cnv:ontresearch/wf-cnv:sha428cb19e51370020ccf29ec2af4eead44c6a17c2, withLabel:wf_human_str:ontresearch/wf-human-variation-str:shaa2f49ce57886426516eadd4048b6fdf9c22c7437, withLabel:snpeff_annotation:ontresearch/snpeff:sha1acac7c26e15955ec29021b28e6ee5a5140c245e, withLabel:wf_common:ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9, withLabel:spectre:ontresearch/spectre:sha659145073b9a1506153d3cb154d9973eabd785e3, default:ontresearch/wf-human-variation:shad3aed855cd007c653b8fc8cb16fe46c90199990f]
  launchDir      : /mnt/HomeApps/Home/bob
  workDir        : /mnt/HomeApps/Home/bob/work
  projectDir     : /data07/tools/nextflow-wf/epi2me-labs/wf-human-variation
  userName       : bob
  profile        : singularity
  configFiles    : /data07/tools/nextflow-wf/epi2me-labs/wf-human-variation/nextflow.config

Workflow Options
  snp            : true

Main options
  sample_name    : DEMO
  bam            : /data10/users/bob/epi2me_test_run/data_raw/wf-human-variation-demo/demo.bam
  ref            : /data10/users/bob/epi2me_test_run/data_raw/wf-human-variation-demo/demo.fasta
  basecaller_cfg : dna_r10.4.1_e8.2_400bps_hac_prom

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-human-variation for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-human-variation v2.0.0-g52e3698.
--------------------------------------------------------------------------------
executor >  local (8)
executor >  local (8)
[3c/33007f] process > bam_ingress:check_for_alignment (1) [100%] 1 of 1, failed: 1 ✘
[-        ] process > bam_ingress:cram_to_bam             -
[-        ] process > bam_ingress:minimap2_alignment      -
[-        ] process > getGenome                           -
[eb/70f3fa] process > cram_cache (1)                      [100%] 1 of 1, failed: 1 ✘
[-        ] process > getAllChromosomesBed                -
[-        ] process > mosdepth_input                      -
[-        ] process > getVersions                         -
[-        ] process > getParams                           -
[-        ] process > get_coverage                        -
[-        ] process > readStats                           -
[-        ] process > makeAlignmentReport                 -
[-        ] process > failedQCReport                      -
[a3/0c8a7d] process > lookup_clair3_model (1)             [100%] 1 of 1, failed: 1 ✘
[-        ] process > snp:make_chunks                     -
[-        ] process > snp:pileup_variants                 -
[-        ] process > snp:aggregate_pileup_variants       -
[-        ] process > snp:select_het_snps                 -
[-        ] process > snp:phase_contig                    -
[-        ] process > snp:get_qual_filter                 -
[-        ] process > snp:create_candidates               -
[-        ] process > snp:evaluate_candidates             -
[-        ] process > snp:aggregate_full_align_variants   -
[-        ] process > snp:merge_pileup_and_full_vars      -
[-        ] process > snp:aggregate_all_variants          -
[-        ] process > annotate_snp_vcf                    -
[-        ] process > concat_snp_vcfs                     -
[-        ] process > sift_clinvar_snp_vcf                -
[-        ] process > vcfStats                            -
[-        ] process > report_snp:getVersions              -
[-        ] process > report_snp:getParams                -
[-        ] process > report_snp:makeReport               -
[-        ] process > output_snp                          -
[-        ] process > configure_jbrowse                   -
[-        ] process > publish_artifact (1)                -
ERROR ~ Error executing process > 'lookup_clair3_model (1)'

Caused by:
  Process `lookup_clair3_model (1)` terminated with an error exit status (255)

Command executed:

  clair3_model=$(resolve_clair3_model.py lookup_table 'dna_r10.4.1_e8.2_400bps_hac_prom')
  cp -r ${CLAIR_MODELS_PATH}/${clair3_model} model
  echo "Basecall model: dna_r10.4.1_e8.2_400bps_hac_prom"
  echo "Clair3 model  : ${clair3_model}"

Command exit status:
  255

Command output:
  (empty)

Command error:
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  FATAL:   Relocation not allowed with starter-suid

Work dir:
  /mnt/HomeApps/Home/bob/work/a3/0c8a7de0e08690192edd8a138a7584

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

no

Other demo data information

No response

SamStudio8 commented 3 months ago

Hi @blanleung, this looks like something to approach your system administrator about in the first instance.

blanleung commented 3 months ago

Hi @SamStudio8, thanks for the suggestion will do.

blanleung commented 3 months ago

I've found the issue. Our data storages are mounted on /mnt/data07 and /mnt/data10. In the nextflow config, singularity automount was set to true, which mean singularity was run with -B /mnt, resulting in the fatal error. Setting the automount to false and binding the data storages with runOptions = "-B /mnt/data07 -B /mnt/data10 --contain" got it to work.