Where to change path:prefix so GEMMAKER runs?

[feltus]

Hi WSU:

(1) Gemmaker is breaking again. I ran:

(base) student@landingvm:~/Desktop/classroom/myfiles/gemmaker-test$ nextflow run systemsgenetics/gemmaker -profile test,singularity

(2) ERROR: " Process GEMmaker:retrieve_sra_metadata terminated with an error exit status (1)"

(3) HOW DO I " # Remove the 'path:' prefix. "?

Thanks, Alex

DETAILS
____________________________-

N E X T F L O W  ~  version 21.10.0
Pulling systemsgenetics/gemmaker ...
 downloaded from https://github.com/SystemsGenetics/GEMmaker.git
Launching `systemsgenetics/gemmaker` [backstabbing_mirzakhani] - revision: 3107932f71 [master]

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  systemsgenetics/gemmaker v2.1.0
------------------------------------------------------
Core Nextflow options
  revision                  : master
  runName                   : backstabbing_mirzakhani
  containerEngine           : singularity
  launchDir                 : /home/student/Desktop/classroom/myfiles/gemmaker-test
  workDir                   : /home/student/Desktop/classroom/myfiles/gemmaker-test/work
  projectDir                : /home/student/.nextflow/assets/systemsgenetics/gemmaker
  userName                  : student
  profile                   : test,singularity
  configFiles               : /home/student/.nextflow/assets/systemsgenetics/gemmaker/nextflow.config

Input/output options
  input                     : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/*{1,2}.fastq
  pipeline                  : hisat2
  sras                      : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/SRA_IDs.txt  skip_samples              : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/samples2skip.txt
  failed_run_report_template: /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/failed_sra_runs.template.html
  multiqc_config_file       : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/multiqc_config.yaml
  multiqc_custom_logo       : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/systemsgenetics-gemmaker_logo.png

Kallisto Pipeline
  kallisto_index_path       : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/references/CORG.transcripts.Kallisto.indexed
  kallisto_bootstrap_samples: 0

Salmon Pipeline
  salmon_index_path         : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/references/CORG.transcripts.Salmon.indexed

Hisat Pipeline
  hisat2_base_name          : CORG
  hisat2_index_dir          : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/references/CORG.genome.Hisat2.indexed
  hisat2_gtf_file           : /home/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/references/CORG.transcripts.gtf

Reference genome options
  igenomes_ignore           : true

Generic options
  publish_dir_mode          : link

Max job request options
  max_cpus                  : 2
  max_memory                : 6 GB
  max_time                  : 2d

Institutional config options
  config_profile_name       : Test profile
  config_profile_description: Minimal test dataset to check pipeline function

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use systemsgenetics/gemmaker for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.3620945

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/systemsgenetics/gemmaker/blob/master/CITATIONS.md
------------------------------------------------------
[-        ] process > GEMmaker:get_software_versions -
[-        ] process > GEMmaker:retrieve_sra_metadata -
executor >  local (1)
[-        ] process > GEMmaker:get_software_versions -
[77/c4a466] process > GEMmaker:retrieve_sra_metadata [  0%] 0 of 1
[-        ] process > GEMmaker:download_runs         -
[-        ] process > GEMmaker:fastq_dump            -
[-        ] process > GEMmaker:fastq_merge           -
[-        ] process > GEMmaker:fastqc_1              -
[-        ] process > GEMmaker:trimmomatic           -
[-        ] process > GEMmaker:fastqc_2              -
[-        ] process > GEMmaker:hisat2                -
[-        ] process > GEMmaker:samtools_sort         -
[-        ] process > GEMmaker:samtools_index        -
executor >  local (1)
[-        ] process > GEMmaker:get_software_versions -
[77/c4a466] process > GEMmaker:retrieve_sra_metadata [100%] 1 of 1, failed: 1 ✘
[-        ] process > GEMmaker:download_runs         -
[-        ] process > GEMmaker:fastq_dump            -
[-        ] process > GEMmaker:fastq_merge           -[-        ] process > GEMmaker:fastqc_1              -
[-        ] process > GEMmaker:trimmomatic           -[-        ] process > GEMmaker:fastqc_2              -
[-        ] process > GEMmaker:hisat2                -
[-        ] process > GEMmaker:samtools_sort         -
[-        ] process > GEMmaker:samtools_index        -
[-        ] process > GEMmaker:stringtie             -
[-        ] process > GEMmaker:stringtie_fpkm_tpm    -
[-        ] process > GEMmaker:next_sample           -
[-        ] process > GEMmaker:multiqc               -
[-        ] process > GEMmaker:create_gem            -
[-        ] process > GEMmaker:failed_run_report     -
[-        ] process > GEMmaker:clean_work_files      -
[-        ] process > GEMmaker:clean_work_dirs       -
Error executing process > 'GEMmaker:retrieve_sra_metadata'

Caused by:
  Process `GEMmaker:retrieve_sra_metadata` terminated with an error exit status (1)
Command executed:

  echo "#TRACE n_remote_run_ids=`cat SRA_IDs.txt | wc -l`"

  # Remove the 'path:' prefix. This was added to prevent
  # Nextflow from recoginzing the path and noticing the work
  # directory changed and trying to re-run this process even
  # if it succeeded.
  workdir=`echo path:/home/student/Desktop/classroom/myfiles/gemmaker-test/work/GEMmaker | sed 's/path://'`

  retrieve_sra_metadata.py       --run_id_file SRA_IDs.txt       --meta_dir "$workdir"       --out_file SRA_run2exp.tsv        --skip_file /home
/student/.nextflow/assets/systemsgenetics/gemmaker/assets/demo/samples2skip.txt

Command exit status:
  1

Command output:
  #TRACE n_remote_run_ids=1

Command error:
  Retrieving metadata for run IDs      0 -     99 of      1...
  Fetching IDs: SRR649944.  Traceback (most recent call last):
    File "/home/student/.nextflow/assets/systemsgenetics/gemmaker/bin/retrieve_sra_metadata.py", line 293, in <module>
      failed_runs = download_runs_meta(missing_runs, meta_dir, args.PAGE_SIZE)
    File "/home/student/.nextflow/assets/systemsgenetics/gemmaker/bin/retrieve_sra_metadata.py", line 68, in download_runs_meta
      response_xml = response_obj.read().decode(response_obj.headers.get_content_charset())
  TypeError: decode() argument 'encoding' must be str, not None

Work dir:
  /home/student/Desktop/classroom/myfiles/gemmaker-test/work/77/c4a466bacd0448c8d39cd6ad67947d

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

WARN: To render the execution DAG in the required format it is required to install Graphviz -- See http://www.graphviz.org for more info.

[spficklin]

Thanks @feltus we noticed this problem just yesterday ourselves. We'll look into it!

[spficklin]

The problem is resolved and there is a new version of GEMmaker. Try testing again, but you'll need to clear your Nextflow cache for GEMmaker. You can use the nextflow pull command to do that, then run your test again:

nextflow pull systemsgenetics/gemmaker
nextflow run systemsgenetics/gemmaker -profile test,singularity

The problem was that NCBI changed the way the web response was returned for metadata and that broke things.