epi2me-labs / wf-single-cell

Other
70 stars 38 forks source link

Epi2Me: Running On Singularity #19

Closed theheking closed 1 year ago

theheking commented 1 year ago

What happened?

I am currently trying to run wf-single-cell-demo via singularity but coming up with this error. Is there a workaround for not using docker and only having singularity installed as I do not have sudo privileges. Thank you in advance,

Operating System

ubuntu 20.04

Workflow Execution

Command line

Workflow Execution - EPI2ME Labs Versions

wf-single-cell-demo

Workflow Execution - CLI Execution Profile

Singularity

Workflow Version

wf-single-cell v0.2.0-g539c102

Relevant log output

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-single-cell v0.2.0-g539c102
--------------------------------------------------------------------------------
Core Nextflow options
  revision        : master
  runName         : big_bhaskara
  containerEngine : docker
  launchDir       : /scratch/
  workDir         : /scratch/output/workspace
  projectDir      : /home/913/.nextflow/assets/epi2me-labs/wf-single-cell
  userName        : 
  profile         : standard
  configFiles     : /home/913/.nextflow/assets/epi2me-labs/wf-single-cell/nextflow.config

Input Options
  fastq           : wf-single-cell-demo/chr17.fq.gz
  ref_genome_dir  : wf-single-cell-demo/
  kit_config      : /home/913/.nextflow/assets/epi2me-labs/wf-single-cell/kit_configs.csv
  expected_cells  : 100

Advanced options
  matrix_min_genes: 200
  umap_plot_genes : /home/913/.nextflow/assets/epi2me-labs/wf-single-cell/umap_plot_genes.csv

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-single-cell for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-single-cell v0.2.0-g539c102.
--------------------------------------------------------------------------------
Checking fastq input.
Single file input detected.
[-        ] process > pipeline:getVersions           -
[-        ] process > pipeline:getParams             -
[-        ] process > pipeline:summariseCatChunkR... -
[-        ] process > pipeline:stranding:call_ada... -
[-        ] process > pipeline:stranding:combine_... -
[-        ] process > pipeline:stranding:summariz... -
[-        ] process > pipeline:align:call_paftools   -
[-        ] process > pipeline:align:get_chrom_sizes -
[-        ] process > pipeline:align:align_to_ref    -
[-        ] process > pipeline:process_bams:split... -
[-        ] process > pipeline:process_bams:get_c... -
[-        ] process > pipeline:process_bams:extra... -
[-        ] process > pipeline:process_bams:gener... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:strin... -
[-        ] process > pipeline:process_bams:align... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:clust... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:annot... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -

[-        ] process > pipeline:getVersions           [  0%] 0 of 1
[-        ] process > pipeline:getParams             [  0%] 0 of 1
[-        ] process > pipeline:summariseCatChunkR... [  0%] 0 of 1
[-        ] process > pipeline:stranding:call_ada... -
[-        ] process > pipeline:stranding:combine_... -
[-        ] process > pipeline:stranding:summariz... -
[-        ] process > pipeline:align:call_paftools   [  0%] 0 of 1
[-        ] process > pipeline:align:get_chrom_sizes [  0%] 0 of 1
[-        ] process > pipeline:align:align_to_ref    -
[-        ] process > pipeline:process_bams:split... [  0%] 0 of 1
[-        ] process > pipeline:process_bams:get_c... -
[-        ] process > pipeline:process_bams:extra... -
[-        ] process > pipeline:process_bams:gener... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:strin... -
[-        ] process > pipeline:process_bams:align... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:clust... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:annot... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -

executor >  local (5)
[a6/729dbd] process > pipeline:getVersions           [  0%] 0 of 1
[92/31d9c0] process > pipeline:getParams             [  0%] 0 of 1
[-        ] process > pipeline:summariseCatChunkR... [  0%] 0 of 1
[-        ] process > pipeline:stranding:call_ada... -
[-        ] process > pipeline:stranding:combine_... -
[-        ] process > pipeline:stranding:summariz... -
[b7/ef71f4] process > pipeline:align:call_paftools   [  0%] 0 of 1
[c2/f34035] process > pipeline:align:get_chrom_sizes [  0%] 0 of 1
[-        ] process > pipeline:align:align_to_ref    -
[42/601ec6] process > pipeline:process_bams:split... [  0%] 0 of 1
[-        ] process > pipeline:process_bams:get_c... -
[-        ] process > pipeline:process_bams:extra... -
[-        ] process > pipeline:process_bams:gener... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:strin... -
[-        ] process > pipeline:process_bams:align... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:clust... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:annot... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -
Error executing process > 'pipeline:align:get_chrom_sizes'

Caused by:
  Process `pipeline:align:get_chrom_sizes` terminated with an error exit status (127)

Command executed:

  cut -f1,2 ref_genome.fai | sort -V > chr_sizes

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.run: line 307: docker: command not found

Work dir:
  /scratch/output/workspace/c2/f3403515b44f594bf29ae1a6967f1d

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

WARN: Killing running tasks (3)

executor >  local (5)
[a6/729dbd] process > pipeline:getVersions           [  0%] 0 of 1
[92/31d9c0] process > pipeline:getParams             [200%] 2 of 1, failed: 2 ✘
[-        ] process > pipeline:summariseCatChunkR... [  0%] 0 of 1
[-        ] process > pipeline:stranding:call_ada... -
[-        ] process > pipeline:stranding:combine_... -
[-        ] process > pipeline:stranding:summariz... -
[b7/ef71f4] process > pipeline:align:call_paftools   [100%] 1 of 1, failed: 1 ✘
[c2/f34035] process > pipeline:align:get_chrom_sizes [100%] 1 of 1, failed: 1 ✘
[-        ] process > pipeline:align:align_to_ref    -
[42/601ec6] process > pipeline:process_bams:split... [100%] 1 of 1, failed: 1 ✘
[-        ] process > pipeline:process_bams:get_c... -
[-        ] process > pipeline:process_bams:extra... -
[-        ] process > pipeline:process_bams:gener... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:strin... -
[-        ] process > pipeline:process_bams:align... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:clust... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:annot... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -
Error executing process > 'pipeline:align:get_chrom_sizes'

Caused by:
  Process `pipeline:align:get_chrom_sizes` terminated with an error exit status (127)

Command executed:

  cut -f1,2 ref_genome.fai | sort -V > chr_sizes

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.run: line 307: docker: command not found

Work dir:
  /scratch/output/workspace/c2/f3403515b44f594bf29ae1a6967f1d

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
nrhorner commented 1 year ago

Hi @theheking

I can see from your log that you were running with the default standard profile (which uses docker) Could you try using -profile singularity

Thanks

theheking commented 1 year ago

Thank you! Did not see the profile flag on help/man pages. Cheers

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-single-cell v0.2.0
--------------------------------------------------------------------------------
Typical pipeline command:

  nextflow run epi2me-labs/wf-single-cell \
    --fastq test_data/reads.fastq.gz

Input Options
  --fastq                        [string]  FASTQ files to use in the analysis.
  --ref_genome_dir               [string]  The path to the 10x reference directory
  --kit_config                   [string]  A file defining the configurations associated with the various supported 10x kits.
  --merge_bam                    [boolean] Merge BAM alignment files into a single BAM file per sample
  --kit_name                     [string]  10x kit name [default: 3prime]
  --kit_version                  [string]  10x kit version [default: v3]
  --expected_cells               [integer] Number of expected cells. [default: 500]

Sample Options
  --single_cell_sample_sheet     [string]  An optional CSV file used to assign library metadata to samples. If all samples have the same library metadata, this
                                           can be supplied instead by using the parameters (kit_name, kit_version, expected cells)
  --sample_sheet                 [string]  A CSV file used to map barcodes to sample aliases. The sample sheet can be provided when the input data is a
                                           directory containing sub-directories with FASTQ files.
  --sample                       [string]  A single sample name for non-multiplexed data. Permissible if passing a single .fastq(.gz) file or directory of
                                           .fastq(.gz) files.

Output Options
  --out_dir                      [string]  Directory for output of all workflow results. [default: output]

Advanced options
  --max_threads                  [integer] This is used to control how many threads the following multithread-capable processes can use: combine_chrom_bams(max
                                           8 threads), stringtie(max_threads/4),  align_to_transcriptome(max_threads/2) .It is also used as the number of parts
                                           to chunk the fastq data into before processing. [default: 4]
  --read_structure_batch_size    [integer] Batch size when processing reads using adapter_scan_vsearch.py (changing not recommended). [default: 40000]
  --barcode_adapter1_suff_length [integer] Suffix length of the read1 adapter to use in creating the probe sequence for identifying barcode/UMI bases.
                                           [default: 10]
  --barcode_min_quality          [integer] Minimum allowed nucleotide-level quality score in the extracted/uncorrected barcode sequence [default: 15]
  --barcode_max_ed               [integer] Maximum allowable edit distance between uncorrected barcode and the best matching corrected barcode from the sample
                                           whitelist. [default: 2]
  --barcode_min_ed_diff          [integer] Minimum allowable edit distance difference between whitelist barcode candidates. [default: 2]
  --gene_assigns_minqv           [integer] Minimum alignment qscore allowed for a read to be assigned to a gene or genomic region. [default: 60]
  --umi_genomic_interval         [integer] Size of genomic window (bp) to assign to a read if alignment falls outside of an annotated gene. [default:
                                           1000]
  --umi_cell_gene_max_reads      [integer] Maximum number of reads to cluster for a particular barcode/gene combination. [default: 20000]
  --umi_cluster_max_threads      [integer] Maximum number of threads to use per-chromosome in UMI clustering step. [default: 4]
  --matrix_min_genes             [integer] Filter cells from the gene expression matrix if they contain fewer than <matrix_min_genes> genes. [default:
                                           25]
  --matrix_min_cells             [integer] Filter genes from the gene expression matrix that are observed in fewer than <matrix_min_cells> cells. [default:
                                           3]
  --matrix_max_mito              [integer] Filter cells from the gene expression matrix if more than <matrix_max_mito> percent of UMI counts come from
                                           mitochondrial genes. [default: 20]
  --matrix_norm_count            [integer] Normalize expression matrix to <matrix_norm_count> counts per cell. [default: 10000]
  --umap_plot_genes              [string]  File containing a list of gene symbols (one symbol per line) to annotate with expression values in the UMAP
                                           projections.
  --resources_mm2_max_threads    [integer] Maximum allowed threads for the minimap2 stage. [default: 4]
  --resources_mm2_flags          [string]  Optional flags for the minimap2 stage. [default: -I 16G]
  --mito_prefix                  [string]  Gene name prefix to identify for mitochondrial genes. [default: MT-]

Miscellaneous Options
  --disable_ping                 [boolean] Enable to prevent sending a workflow ping.
  --help                         [boolean] Display help text.
  --version                      [boolean] Display version and exit.

Other parameters
  --process_label                [string]  The main process label for template processes to use by default [default: singlecell]
  --aws_image_prefix             [string]  null
  --aws_queue                    [string]  null
  --monochrome_logs              [boolean] null
  --validate_params              [boolean] null [default: true]
  --show_hidden_params           [boolean] null