ERROR ~ fromIndex = -1

[Adrian-Zet]

Description of the bug

Dear Nf-core/Methylseq Community,

In the past 2 weeks, I've tried running the methylseq pipeline on a dataset of patients with depression vs normal from Bisulfite Methylation.

I've ran into multiple problems, most had something to do with Docker creating folders on my Azure VM (the one I setup to run nf-core/methylseq). I will list some of the previous errors that I managed to solve at the end of the report here.

Yet, I must mention that nextflow seems to be running good outside of this pipeline, having tested it already in using nf-core/rnaseq , nf-core/smallrnaseq and nf-core/differentialabundance.

Also, for this pipeline, not even the test ran properly, all the errors below resulted from trying to run the test profile too.

Next, I will present the errors, starting with the latest, and moving down towards the earlier errors: ERROR (20-Nov-2023):

ERROR ~ fromIndex = -1 
 -- Check script '/home/saia/.nextflow/assets/nf-core/methylseq/./workflows/methylseq.nf' at line: 102 or see '.nextflow.log' file for more details

Error description from chat GPT: The error message you're encountering, ERROR ~ fromIndex = -1, typically indicates an issue with list or array operations in a Groovy/Java context, such as attempting to access an element from an invalid index in a list or array. In Groovy, which is the scripting language used in Nextflow, negative indices are not valid for list access unless they are used for specific purposes like counting from the end of the list. Analysis: The error sugests it is coming from line 102 of file methylseq.nf. The specific line is meta_clone.id = metaclone.id.split('')[0..-2].join('_') and its context implies that it is handling the input file.

 // Create input channel from input file provided through params.input
    //
    Channel
        .fromSamplesheet("input")
        .map {
            meta, fastq_1, fastq_2 ->
            if (!fastq_2) {
                return [ meta + [ single_end:true ], [ fastq_1 ] ]
            } else {
                return [ meta + [ single_end:false ], [ fastq_1, fastq_2 ] ]
            }
        }
        .groupTuple()
        .map {
            meta, fastq ->
            def meta_clone = meta.clone()
            meta_clone.id = meta_clone.id.split('_')[0..-2].join('_')
            [ meta_clone, fastq ]
        }
        .groupTuple(by: [0])
        .branch {
            meta, fastq ->
            single: fastq.size() == 1
            return [ meta, fastq.flatten() ]
            multiple: fastq.size() > 1
            return [ meta, fastq.flatten() ]
        }
        .set { ch_fastq }

ERROR #2 (17-Nov-2023):

Workflow execution completed unsuccessfully
The exit status of the task that caused the workflow execution to fail was: 127

Error executing process > 'NFCORE_METHYLSEQ:METHYLSEQ:PREPARE_GENOME:BISMARK_GENOMEPREPARATION (BismarkIndex/genome.fa)'

Caused by:
  Process `NFCORE_METHYLSEQ:METHYLSEQ:PREPARE_GENOME:BISMARK_GENOMEPREPARATION (BismarkIndex/genome.fa)` terminated with an error exit status (127)

Command executed:

  bismark_genome_preparation \
      --bowtie2 \
      BismarkIndex

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_METHYLSEQ:METHYLSEQ:PREPARE_GENOME:BISMARK_GENOMEPREPARATION":
      bismark: $(echo $(bismark -v 2>&1) | sed 's/^.*Bismark Version: v//; s/Copyright.*$//')
  END_VERSIONS

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.sh: line 3: bismark_genome_preparation: command not found

Work dir:
  /home/saia/work/22/b1a9b75491e3b52812dc6f29789340

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

Update: I did manage to run the test pipeline once, but it did fail 2 steps, and then I went back to the errors I've described above. I will attach the log from the runs.

I will greatly appreciate any form of help.

Thank you all very much! Adrian

Command used and terminal output

#this was the command that resulted in ERROR#1:
nextflow run nf-core/methylseq
         --input METHYL_MDD_CTRL_S-MDD_corrected2.csv
         --outdir METHYL_MDD_CTRL_S-MDD
         --genome GRCh37
         --aligner bismark
         -profile docker
         -with-tower

#this is the command that resulted in ERROR#2:
nextflow run nf-core/methylseq
         -profile test
         --outdir test-meth
         -with-tower

Relevant files

logs.zip

System information

Nextflow version (eg. 23.10.0) Hardware: Azure VM with configurations of 32 cores / 128 GB & 64 cores / 256 GB Executor local Container engine: Docker OS Ubuntu 20.04 Version of nf-core/methylseq: latest pulled from the website

[wassimsalam01]

Edit: added params file and attached log file

Just ran into this issue too. Commenting to bump it up.

ERROR ~ fromIndex = -1

 -- Check script '/home/salaw97/.nextflow/assets/nf-core/methylseq/./workflows/methylseq.nf' at line: 102 or see '.nextflow.log' file for more details

when executing sbatch my-workflow.sh

and here is the script

#!/bin/bash

#SBATCH --mail-user=<my-email>
#SBATCH --mail-type=FAIL
#SBATCH --job-name=pulex
#SBATCH --mem=1GB
#SBATCH --partition=main
#SBATCH --nodes=1
#SBATCH --qos=hiprio
#SBATCH --time=0-03:00:00
#SBATCH --error=slurm/error/%x.%j.err
#SBATCH --output=slurm/output/%x.%j.out

module purge
module load Nextflow/23.04.2

nextflow run nf-core/methylseq \
    -r 2.5.0 \
    -profile fub_curta \
    -params-file ./nf-params_bismark_pulex.yaml

and params-file:

input: '../epi_data/samplesheet_test.csv'
outdir: './results'
email: '<my-email>'
save_reference: true
save_align_intermeds: true
unmapped: true
save_trimmed: true
fasta: '../genomes/daphnia_pulex.fasta'
comprehensive: true
zymo: true
cytosine_report: true

Nextflow version (23.04.2) Hardware: HPC Executor: Slurm Container engine: Singularity OS: PuTTY methylseq version: 2.5.0

.nextflow.log

[edmundmiller]

I thought #339 would've fixed this but I guess not 🙃

We should probably catch it at validation time is the best place to check.

We could also add a check right here https://github.com/nf-core/methylseq/blob/0c464660d92f6d1e8ac9e8a9f16de297665aa62e/workflows/methylseq.nf#L102

[pjm43]

I've also run into the same issue. Is there a new release that fixes this issue: ERROR ~ fromIndex = -1 -- Check script '/fslhome/pjm43/fsl_groups/fslg_lifesciences/pjm43/nf-core_download/nf-core-methylseq_2.5.0/2_5_0/./workflows/methylseq.nf' at line: 102 or see '.nextflow.log' file for more details

I see that there is a pull request that will close this issue. Does that mean there is a new version of methylseq that fixes this issue? If so, how do I access it?

Thanks in advance!

[mpiersonsmela]

I am also getting this problem. Is there a release where it's fixed?

[sateeshperi]

we are working on a release soon...you can try pulling the dev branch and let us know if this issue has been fixed

[gianfilippo]

same here. I am trying the dev branch and the error is no longer there.

Thanks!