search

PGScatalog / pgsc_calc

The Polygenic Score Catalog Calculator is a nextflow pipeline for polygenic score calculation
https://pgsc-calc.readthedocs.io/en/latest/
Apache License 2.0
117 stars 21 forks source link

INTERSECT_VARIANTS error with run_ancestry #319

Closed samreenzafer closed 4 months ago

samreenzafer commented 4 months ago

Hi I was successfully able to run 2 PGS traits for my plink format target data (~20K EUR samples).

However when I try to run the same with ancestry adjustment, I am getting an error at the INTERSECT_VARIANTS step. I tried making the hwe, geno and maf thresholds more relaxed but the error persists. I can't identify what the issue maybe. 


[zafers02@lc02a27 test_nextflow_1]$ nextflow run pgscatalog/pgsc_calc --max_cpus 1 -profile singularity --input biome_cnics_plink/samplesheet.BiomeCNICS.csv --pgs_id PGS003446,PGS002237 --target_build GRCh37 -w /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work/  --genotypes_cache /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache --min_overlap 0.20 --run_ancestry /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/resources/pgsc_HGDP+1kGP_v1.tar.zst  -resume --hwe_ref 1e-10 --maf_ref 0.01 --geno_ref 0.5

 N E X T F L O W   ~  version 24.04.2

Launching `https://github.com/pgscatalog/pgsc_calc` [loving_raman] DSL2 - revision: 45326ac066 [main]

WARN: Access to undefined parameter `monochromeLogs` -- Initialise it to a default value eg. `params.monochromeLogs = some_value`

------------------------------------------------------
  pgscatalog/pgsc_calc v2.0.0-alpha.6-g45326ac
------------------------------------------------------
Core Nextflow options
  revision          : main
  runName           : loving_raman
  containerEngine   : singularity
  launchDir         : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1
  workDir           : /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work
  projectDir        : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc
  userName          : zafers02
  profile           : singularity
  configFiles       : 

Input/output options
  input             : biome_cnics_plink/samplesheet.BiomeCNICS.csv
  pgs_id            : PGS003446,PGS002237
  genotypes_cache   : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache
  outdir            : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/results

Reference options
  run_ancestry      : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/resources/pgsc_HGDP+1kGP_v1.tar.zst
  ref_samplesheet   : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/reference.csv
  geno_ref          : 0.5
  maf_ref           : 0.01
  hwe_ref           : 1e-10
  ld_grch37         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg19-GRCh37.txt
  ld_grch38         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg38-GRCh38.txt
  ancestry_checksums: /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/checksums.txt

Compatibility options
  target_build      : GRCh37

Matching options
  min_overlap       : 0.2

Max job request options
  max_cpus          : 1

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use pgscatalog/pgsc_calc for your analysis please cite:

* The Polygenic Score Catalog
  https://doi.org/10.1038/s41588-021-00783-5

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/pgscatalog/pgsc_calc/blob/master/CITATIONS.md

Reference database provided: skipping bootstrap
executor >  local (3)
[3b/2ecd18] PGS…PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:]) [100%] 1 of 1 ✔
[b0/7e1cda] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                               [  0%] 0 of 1
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)    [100%] 1 of 1, stored: 1 ✔
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                       -
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                            [100%] 1 of 1, stored: 1 ✔
[af/595670] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)  [  0%] 0 of 1
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET                           -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS                                             -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ                                        -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE                                      -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS                                             -
ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)'

Caused by:
  Process `PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)` terminated with an error exit status (1)

Command executed:

  pgscatalog-intersect --ref GRCh37_HGDP+1kGP_ALL.pvar.zst         --target GRCh37_BiomeCnics_ALL.bim.zst         --chrom ALL         --maf_target 0.1         --geno_miss 0.1         --outdir .         -v

  n_matched=$(sed -n '3p' intersect_counts_ALL.txt)

  if [ $n_matched == "0" ]
  then
      echo "ERROR: No variants in intersection"
      exit 1
  else
      mv matched_variants.txt.gz BiomeCnics_ALL_matched.txt.gz
  fi

  cat <<-END_VERSIONS > versions.yml
  INTERSECT_VARIANTS:
      pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.__version__)'))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:45:58 INFO     Processed 66000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:02 INFO     Processed 66500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:07 INFO     Processed 67000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:11 INFO     Processed 67500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:15 INFO     Processed 68000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:20 INFO     Processed 68500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:24 INFO     Processed 69000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:28 INFO     Processed 69500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:32 INFO     Processed 70000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:37 INFO     Processed 70500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:41 INFO     Processed 71000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:45 INFO     Processed 71500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:50 INFO     Processed 72000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:54 INFO     Processed 72500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:58 INFO     Processed 73000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:03 INFO     Processed 73500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:07 INFO     Processed 74000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:12 INFO     Processed 74500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:16 INFO     Processed 75000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:20 INFO     Processed 75500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:24 INFO     Processed 76000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:29 INFO     Processed 76500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:33 INFO     Processed 77000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:37 INFO     Processed 77500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:41 INFO     Processed 78000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:46 INFO     Processed 78500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:50 INFO     Processed 79000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:54 INFO     Processed 79500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:59 INFO     Processed 80000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:03 INFO     Processed 80500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:07 INFO     Processed 81000000 REFERENCE variants
executor >  local (3)
[3b/2ecd18] PGS…PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:]) [100%] 1 of 1 ✔
[b0/7e1cda] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                               [  0%] 0 of 1
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)    [100%] 1 of 1, stored: 1 ✔
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                       -
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                            [100%] 1 of 1, stored: 1 ✔
[af/595670] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)  [100%] 1 of 1, failed: 1 ✘
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET                           -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS                                             -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ                                        -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE                                      -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS                                             -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)'

Caused by:
  Process `PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)` terminated with an error exit status (1)

Command executed:

  pgscatalog-intersect --ref GRCh37_HGDP+1kGP_ALL.pvar.zst         --target GRCh37_BiomeCnics_ALL.bim.zst         --chrom ALL         --maf_target 0.1         --geno_miss 0.1         --outdir .         -v

  n_matched=$(sed -n '3p' intersect_counts_ALL.txt)

  if [ $n_matched == "0" ]
  then
      echo "ERROR: No variants in intersection"
      exit 1
  else
      mv matched_variants.txt.gz BiomeCnics_ALL_matched.txt.gz
  fi

  cat <<-END_VERSIONS > versions.yml
  INTERSECT_VARIANTS:
      pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.__version__)'))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:45:58 INFO     Processed 66000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:02 INFO     Processed 66500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:07 INFO     Processed 67000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:11 INFO     Processed 67500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:15 INFO     Processed 68000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:20 INFO     Processed 68500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:24 INFO     Processed 69000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:28 INFO     Processed 69500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:32 INFO     Processed 70000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:37 INFO     Processed 70500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:41 INFO     Processed 71000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:45 INFO     Processed 71500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:50 INFO     Processed 72000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:54 INFO     Processed 72500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:58 INFO     Processed 73000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:03 INFO     Processed 73500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:07 INFO     Processed 74000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:12 INFO     Processed 74500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:16 INFO     Processed 75000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:20 INFO     Processed 75500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:24 INFO     Processed 76000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:29 INFO     Processed 76500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:33 INFO     Processed 77000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:37 INFO     Processed 77500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:41 INFO     Processed 78000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:46 INFO     Processed 78500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:50 INFO     Processed 79000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:54 INFO     Processed 79500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:59 INFO     Processed 80000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:03 INFO     Processed 80500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:07 INFO     Processed 81000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:11 INFO     Processed 81500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:16 INFO     Processed 82000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:20 INFO     Processed 82500000 REFERENCE variants
executor >  local (3)
[3b/2ecd18] PGS…PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:]) [100%] 1 of 1 ✔
[b0/7e1cda] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                               [  0%] 0 of 1
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)    [100%] 1 of 1, stored: 1 ✔
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                       -
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                            [100%] 1 of 1, stored: 1 ✔
[af/595670] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)  [100%] 1 of 1, failed: 1 ✘
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET                           -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS                                             -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ                                        -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE                                      -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS                                             -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)'

Caused by:
  Process `PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)` terminated with an error exit status (1)

Command executed:

  pgscatalog-intersect --ref GRCh37_HGDP+1kGP_ALL.pvar.zst         --target GRCh37_BiomeCnics_ALL.bim.zst         --chrom ALL         --maf_target 0.1         --geno_miss 0.1         --outdir .         -v

  n_matched=$(sed -n '3p' intersect_counts_ALL.txt)

  if [ $n_matched == "0" ]
  then
      echo "ERROR: No variants in intersection"
      exit 1
  else
      mv matched_variants.txt.gz BiomeCnics_ALL_matched.txt.gz
  fi

  cat <<-END_VERSIONS > versions.yml
  INTERSECT_VARIANTS:
      pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.__version__)'))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:45:58 INFO     Processed 66000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:02 INFO     Processed 66500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:07 INFO     Processed 67000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:11 INFO     Processed 67500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:15 INFO     Processed 68000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:20 INFO     Processed 68500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:24 INFO     Processed 69000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:28 INFO     Processed 69500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:32 INFO     Processed 70000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:37 INFO     Processed 70500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:41 INFO     Processed 71000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:45 INFO     Processed 71500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:50 INFO     Processed 72000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:54 INFO     Processed 72500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:58 INFO     Processed 73000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:03 INFO     Processed 73500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:07 INFO     Processed 74000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:12 INFO     Processed 74500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:16 INFO     Processed 75000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:20 INFO     Processed 75500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:24 INFO     Processed 76000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:29 INFO     Processed 76500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:33 INFO     Processed 77000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:37 INFO     Processed 77500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:41 INFO     Processed 78000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:46 INFO     Processed 78500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:50 INFO     Processed 79000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:54 INFO     Processed 79500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:59 INFO     Processed 80000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:03 INFO     Processed 80500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:07 INFO     Processed 81000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:11 INFO     Processed 81500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:16 INFO     Processed 82000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:20 INFO     Processed 82500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:24 INFO     Processed 83000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Processed 83500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Processed 83552519 REFERENCE variants
executor >  local (3)
[3b/2ecd18] PGS…PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:]) [100%] 1 of 1 ✔
[b0/7e1cda] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                               [  0%] 0 of 1
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)    [100%] 1 of 1, stored: 1 ✔
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                       -
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                            [100%] 1 of 1, stored: 1 ✔
[af/595670] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)  [100%] 1 of 1, failed: 1 ✘
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET                           -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS                                             -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ                                        -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE                                      -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS                                             -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)'

Caused by:
  Process `PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)` terminated with an error exit status (1)

Command executed:

  pgscatalog-intersect --ref GRCh37_HGDP+1kGP_ALL.pvar.zst         --target GRCh37_BiomeCnics_ALL.bim.zst         --chrom ALL         --maf_target 0.1         --geno_miss 0.1         --outdir .         -v

  n_matched=$(sed -n '3p' intersect_counts_ALL.txt)

  if [ $n_matched == "0" ]
  then
      echo "ERROR: No variants in intersection"
      exit 1
  else
      mv matched_variants.txt.gz BiomeCnics_ALL_matched.txt.gz
  fi

  cat <<-END_VERSIONS > versions.yml
  INTERSECT_VARIANTS:
      pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.__version__)'))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:45:58 INFO     Processed 66000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:02 INFO     Processed 66500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:07 INFO     Processed 67000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:11 INFO     Processed 67500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:15 INFO     Processed 68000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:20 INFO     Processed 68500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:24 INFO     Processed 69000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:28 INFO     Processed 69500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:32 INFO     Processed 70000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:37 INFO     Processed 70500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:41 INFO     Processed 71000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:45 INFO     Processed 71500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:50 INFO     Processed 72000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:54 INFO     Processed 72500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:58 INFO     Processed 73000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:03 INFO     Processed 73500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:07 INFO     Processed 74000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:12 INFO     Processed 74500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:16 INFO     Processed 75000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:20 INFO     Processed 75500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:24 INFO     Processed 76000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:29 INFO     Processed 76500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:33 INFO     Processed 77000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:37 INFO     Processed 77500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:41 INFO     Processed 78000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:46 INFO     Processed 78500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:50 INFO     Processed 79000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:54 INFO     Processed 79500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:59 INFO     Processed 80000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:03 INFO     Processed 80500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:07 INFO     Processed 81000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:11 INFO     Processed 81500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:16 INFO     Processed 82000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:20 INFO     Processed 82500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:24 INFO     Processed 83000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Processed 83500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Processed 83552519 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Outputting REFERNCE variants -> reference_variants.txt.gz
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:53:16 INFO     Reading TARGET variants: GRCh37_BiomeCnics_ALL.bim.zst
  Traceback (most recent call last):
    File "/usr/local/bin/pgscatalog-intersect", line 8, in <module>
executor >  local (3)
[3b/2ecd18] PGS…PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:]) [100%] 1 of 1 ✔
[b0/7e1cda] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                               [  0%] 0 of 1
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)    [100%] 1 of 1, stored: 1 ✔
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                       -
[skipped  ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                            [100%] 1 of 1, stored: 1 ✔
[af/595670] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)  [100%] 1 of 1, failed: 1 ✘
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED                               -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET                           -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA                                     -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT                                 -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS                                             -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES                                   -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ                                        -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE                                      -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS                                         -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT                                              -
[-        ] PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS                                             -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)'

Caused by:
  Process `PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)` terminated with an error exit status (1)

Command executed:

  pgscatalog-intersect --ref GRCh37_HGDP+1kGP_ALL.pvar.zst         --target GRCh37_BiomeCnics_ALL.bim.zst         --chrom ALL         --maf_target 0.1         --geno_miss 0.1         --outdir .         -v

  n_matched=$(sed -n '3p' intersect_counts_ALL.txt)

  if [ $n_matched == "0" ]
  then
      echo "ERROR: No variants in intersection"
      exit 1
  else
      mv matched_variants.txt.gz BiomeCnics_ALL_matched.txt.gz
  fi

  cat <<-END_VERSIONS > versions.yml
  INTERSECT_VARIANTS:
      pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.__version__)'))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:45:58 INFO     Processed 66000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:02 INFO     Processed 66500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:07 INFO     Processed 67000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:11 INFO     Processed 67500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:15 INFO     Processed 68000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:20 INFO     Processed 68500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:24 INFO     Processed 69000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:28 INFO     Processed 69500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:32 INFO     Processed 70000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:37 INFO     Processed 70500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:41 INFO     Processed 71000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:45 INFO     Processed 71500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:50 INFO     Processed 72000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:54 INFO     Processed 72500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:46:58 INFO     Processed 73000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:03 INFO     Processed 73500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:07 INFO     Processed 74000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:12 INFO     Processed 74500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:16 INFO     Processed 75000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:20 INFO     Processed 75500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:24 INFO     Processed 76000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:29 INFO     Processed 76500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:33 INFO     Processed 77000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:37 INFO     Processed 77500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:41 INFO     Processed 78000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:46 INFO     Processed 78500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:50 INFO     Processed 79000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:54 INFO     Processed 79500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:47:59 INFO     Processed 80000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:03 INFO     Processed 80500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:07 INFO     Processed 81000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:11 INFO     Processed 81500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:16 INFO     Processed 82000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:20 INFO     Processed 82500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:24 INFO     Processed 83000000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Processed 83500000 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Processed 83552519 REFERENCE variants
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Outputting REFERNCE variants -> reference_variants.txt.gz
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:53:16 INFO     Reading TARGET variants: GRCh37_BiomeCnics_ALL.bim.zst
  Traceback (most recent call last):
    File "/usr/local/bin/pgscatalog-intersect", line 8, in <module>
      sys.exit(run_intersect())
               ^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/pgscatalog/match/cli/intersect_cli.py", line 112, in run_intersect
      if all([v["ID"], freq["ID"], miss["#ID"]]) is False:
                                   ~~~~^^^^^^^
  KeyError: '#ID'
  cp: ‘.command.out’ and ‘.command.out’ are the same file
  cp: ‘.command.err’ and ‘.command.err’ are the same file
  cp: ‘.command.trace’ and ‘.command.trace’ are the same file

Work dir:
  /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work/af/5956700faffd09ad0c829928731510

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details
ERROR ~ ERROR: No results report written!

 -- Check '.nextflow.log' file for details
ERROR ~ ERROR: Projection subworkflow failed

 -- Check '.nextflow.log' file for details
ERROR ~ ERROR: Matching subworkflow failed

 -- Check '.nextflow.log' file for details
ERROR ~ ERROR: No scores calculated!

 -- Check '.nextflow.log' file for details

I'm sorry for the long log, it looks like nextflow made a few re-attempts before actually crashing. I can't figure out what this error is indicating.


  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:48:29 INFO     Outputting REFERNCE variants -> reference_variants.txt.gz
  pgscatalog.match.cli.intersect_cli: 2024-06-18 20:53:16 INFO     Reading TARGET variants: GRCh37_BiomeCnics_ALL.bim.zst
  Traceback (most recent call last):
    File "/usr/local/bin/pgscatalog-intersect", line 8, in <module>
      sys.exit(run_intersect())
               ^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/pgscatalog/match/cli/intersect_cli.py", line 112, in run_intersect
      if all([v["ID"], freq["ID"], miss["#ID"]]) is False:
                                   ~~~~^^^^^^^
  KeyError: '#ID'
  cp: ‘.command.out’ and ‘.command.out’ are the same file
  cp: ‘.command.err’ and ‘.command.err’ are the same file
  cp: ‘.command.trace’ and ‘.command.trace’ are the same file
samreenzafer commented 4 months ago

Just for reference, my successful run without ancestry adjustment was as follows:


[zafers02@lc02a27 test_nextflow_1]$ nextflow run pgscatalog/pgsc_calc --max_cpus 1 -profile singularity --input biome_cnics_plink/samplesheet.BiomeCNICS.csv --pgs_id PGS003446,PGS002237 --target_build GRCh37 -w /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work/  --genotypes_cache /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache --min_overlap 0.20  

 N E X T F L O W   ~  version 24.04.2

Launching `https://github.com/pgscatalog/pgsc_calc` [fervent_aryabhata] DSL2 - revision: 45326ac066 [main]

WARN: Access to undefined parameter `monochromeLogs` -- Initialise it to a default value eg. `params.monochromeLogs = some_value`

------------------------------------------------------
  pgscatalog/pgsc_calc v2.0.0-alpha.6-g45326ac
------------------------------------------------------
Core Nextflow options
  revision          : main
  runName           : fervent_aryabhata
  containerEngine   : singularity
  launchDir         : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1
  workDir           : /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work
  projectDir        : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc
  userName          : zafers02
  profile           : singularity
  configFiles       : 

Input/output options
  input             : biome_cnics_plink/samplesheet.BiomeCNICS.csv
  pgs_id            : PGS003446,PGS002237
  genotypes_cache   : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache
  outdir            : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/results

Reference options
  ref_samplesheet   : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/reference.csv
  ld_grch37         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg19-GRCh37.txt
  ld_grch38         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg38-GRCh38.txt
  ancestry_checksums: /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/checksums.txt

Compatibility options
  target_build      : GRCh37

Matching options
  min_overlap       : 0.2

Max job request options
  max_cpus          : 1

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use pgscatalog/pgsc_calc for your analysis please cite:

* The Polygenic Score Catalog
  https://doi.org/10.1038/s41588-021-00783-5

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/pgscatalog/pgsc_calc/blob/master/CITATIONS.md

executor >  local (8)
[e3/51c198] process > PGSCATALOG_PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:])     [100%] 1 of 1 ✔
[60/8d39fe] process > PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                                          [100%] 1 of 1 ✔
[skipped  ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)               [100%] 1 of 1, stored: 1 ✔
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                                          -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                                  -
[48/227c39] process > PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (BiomeCnics chromosome ALL)                            [100%] 1 of 1 ✔
[99/640304] process > PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE (BiomeCnics)                                            [100%] 1 of 1 ✔
[21/ad76ab] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE (BiomeCnics chromosome ALL effect type additive 0) [100%] 1 of 1 ✔
[8a/faa218] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE (BiomeCnics)                                    [100%] 1 of 1 ✔
[51/e37736] process > PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT (BiomeCnics)                                            [100%] 1 of 1 ✔
[b9/1c949a] process > PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS (1)                                                    [100%] 1 of 1 ✔
-[pgscatalog/pgsc_calc] Pipeline completed successfully-
Completed at: 18-Jun-2024 18:04:08
Duration    : 3m 11s
CPU hours   : (a few seconds)
Succeeded   : 8
nebfield commented 4 months ago

I see you've adjusted --min_overlap. Does the standard run complete OK if you don't adjust it? Adjusting it to get the workflow to complete can be a problem.

Are your target genomes imputed? If not the intersection could fail. I'll double check the intersection process with some plink1 (bed/bim/fam) inputs.

samreenzafer commented 4 months ago

Thank you for your reply The overlap with one of the traits was 23% and the other about 35%. That’s why adjusted the min-overlap. My target data is comprised of 2 datasets (one control and one case) which were imputed separately, and then merged on intersecting snp list. I suppose this reduces the overlap #snps, and consequently the overlap with Trait data.

Would you suggest I run the workflow on each dataset separately, to get the best PGS scores for each dataset?

Hope that helps.

Regards,

Samreen

On Wed, Jun 19, 2024 at 1:37 AM Benjamin Wingfield @.***> wrote:

I see you've adjusted --min_overlap. Does the standard run complete OK if you don't adjust it? Adjusting it to get the workflow to complete can be a problem. https://pgsc-calc.readthedocs.io/en/dev/explanation/match.html#adjusting-min-overlap-is-a-bad-idea

Are your target genomes imputed? If not the intersection could fail. I'll double check the intersection process with some plink1 (bed/bim/fam) inputs.

— Reply to this email directly, view it on GitHub https://github.com/PGScatalog/pgsc_calc/issues/319#issuecomment-2178096089, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADLD65BTYYHN7F3EIAAECDZIE7ONAVCNFSM6AAAAABJRBXAIGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZYGA4TMMBYHE . You are receiving this because you authored the thread.Message ID: @.***>

nebfield commented 4 months ago

Could you try again with the most recent release? There was a problem with plink bfile input to pgscatalog-intersect.

I'm not sure why imputation hasn't helped. How many variants do you have in your target genomes total?

https://pgsc-calc.readthedocs.io/en/latest/how-to/change_version.html

samreenzafer commented 4 months ago

Hi Benjamin, My plink file has ~4 Million variants on autosomes.

I am now trying with the latest version, and so far it is stuck at the "INTERSECT_VARIANT" module, do you know how long this module usually takes? 


export NXF_SINGULARITY_CACHEDIR="/sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/work/singularity/"
nextflow run pgscatalog/pgsc_calc --max_cpus 1 -profile singularity --input biome_cnics_plink/samplesheet.BiomeCNICS.csv --pgs_id PGS003446,PGS002237 --target_build GRCh37 -w /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work/  --genotypes_cache /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache --min_overlap 0.20 --run_ancestry /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/resources/pgsc_HGDP+1kGP_v1.tar.zst  -resume -latest 

 N E X T F L O W   ~  version 24.04.2

Pulling pgscatalog/pgsc_calc ...
 Already-up-to-date
Launching `https://github.com/pgscatalog/pgsc_calc` [agitated_wiles] DSL2 - revision: ccfd6367d5 [main]

------------------------------------------------------
  pgscatalog/pgsc_calc v2.0.0-beta-gccfd636
------------------------------------------------------
Core Nextflow options
  revision          : main
  runName           : agitated_wiles
  containerEngine   : singularity
  launchDir         : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1
  workDir           : /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work
  projectDir        : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc
  userName          : zafers02
  profile           : singularity
  configFiles       : 

Input/output options
  input             : biome_cnics_plink/samplesheet.BiomeCNICS.csv
  pgs_id            : PGS003446,PGS002237
  genotypes_cache   : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache
  outdir            : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/results

Reference options
  run_ancestry      : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/resources/pgsc_HGDP+1kGP_v1.tar.zst
  ref_samplesheet   : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/reference.csv
  ld_grch37         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg19-GRCh37.txt
  ld_grch38         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg38-GRCh38.txt
  ancestry_checksums: /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/checksums.txt

Compatibility options
  target_build      : GRCh37

Matching options
  min_overlap       : 0.2

Max job request options
  max_cpus          : 1

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use pgscatalog/pgsc_calc for your analysis please cite:

* The Polygenic Score Catalog
  https://doi.org/10.1101/2024.05.29.24307783
  https://doi.org/10.1038/s41588-021-00783-5

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/pgscatalog/pgsc_calc/blob/main/CITATIONS.md

Reference database provided: skipping bootstrap
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES                                           -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES                                           -
executor >  local (1)
executor >  local (1)
executor >  local (2)
executor >  local (2)
[dd/496c4b] process > PGSCATALOG_PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:]) [100%] 1 of 1 ✔
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES                                          [  0%] 0 of 1
[skipped  ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)           [100%] 1 of 1, stored: 1 ✔
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                                      -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                              -
[skipped  ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                                   [100%] 1 of 1, stored: 1 ✔
[6e/3b81af] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)         [  0%] 0 of 1
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS                                        -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF                                     -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED                                      -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS                                            -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET                                  -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT                                          -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA                                            -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT                                        -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS                                                    -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE                                                     -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES                                          -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ                                               -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE                                                -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE                                             -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS                                                -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT                                                     -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS                                                    -
Pulling Singularity image oras://ghcr.io/pgscatalog/pygscatalog:pgscatalog-utils-1.1.2-singularity [cache /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/work/singularity/ghcr.io-pgscatalog-pygscatalog-pgscatalog-utils-1.1.2-singularity.img]
samreenzafer commented 4 months ago

just to follow up. the pgs-utils singularity image has already finished downloading, even though the output log seems to have stuck at that point. 


[zafers02@li03c03 test_nextflow_1]$ ll /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/work/singularity/ghcr.io-pgscatalog-pygscatalog-pgscatalog-utils-1.1.2-singularity.img 
-rwxrwxrwx 1 zafers02 nicolp01a 521M Jun 20 17:34 /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/work/singularity/ghcr.io-pgscatalog-pygscatalog-pgscatalog-utils-1.1.2-singularity.img
samreenzafer commented 4 months ago

Soit did finish with an error in the 2nd to last module of SCORE_REPORT, and looks like it is a permission issue to some directory. How would you suggest I adjust any parameters to not run into this error? 

[zafers02@lc02a30 test_nextflow_1]$ export NXF_SINGULARITY_CACHEDIR="/sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/work/singularity/"
[zafers02@lc02a30 test_nextflow_1]$ nextflow run pgscatalog/pgsc_calc --max_cpus 1 -profile singularity --input biome_cnics_plink/samplesheet.BiomeCNICS.csv --pgs_id PGS003446,PGS002237 --target_build GRCh37 -w /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work/  --genotypes_cache /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache --min_overlap 0.20 --run_ancestry /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/resources/pgsc_HGDP+1kGP_v1.tar.zst  -resume -latest 

 N E X T F L O W   ~  version 24.04.2

Pulling pgscatalog/pgsc_calc ...
 Already-up-to-date
Launching `https://github.com/pgscatalog/pgsc_calc` [agitated_wiles] DSL2 - revision: ccfd6367d5 [main]

------------------------------------------------------
  pgscatalog/pgsc_calc v2.0.0-beta-gccfd636
------------------------------------------------------
Core Nextflow options
  revision          : main
  runName           : agitated_wiles
  containerEngine   : singularity
  launchDir         : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1
  workDir           : /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work
  projectDir        : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc
  userName          : zafers02
  profile           : singularity
  configFiles       : 

Input/output options
  input             : biome_cnics_plink/samplesheet.BiomeCNICS.csv
  pgs_id            : PGS003446,PGS002237
  genotypes_cache   : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache
  outdir            : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/results

Reference options
  run_ancestry      : /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/resources/pgsc_HGDP+1kGP_v1.tar.zst
  ref_samplesheet   : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/reference.csv
  ld_grch37         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg19-GRCh37.txt
  ld_grch38         : /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/high-LD-regions-hg38-GRCh38.txt
  ancestry_checksums: /hpc/users/zafers02/.nextflow/assets/pgscatalog/pgsc_calc/assets/ancestry/checksums.txt

Compatibility options
  target_build      : GRCh37

Matching options
  min_overlap       : 0.2

Max job request options
  max_cpus          : 1

[dd/496c4b] process > PGSCATALOG_PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:])    [100%] 1 of 1 ✔
[86/c7ad10] process > PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                                         [100%] 1 of 1 ✔
[skipped  ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)              [100%] 1 of 1, stored: 1 ✔
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                                         -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                                 -
[skipped  ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                                      [100%] 1 of 1, stored: 1 ✔
executor >  local (21)
[dd/496c4b] process > PGSCATALOG_PGSCCALC:PGSCCALC:DOWNLOAD_SCOREFILES ([pgs_id:PGS003446 PGS002237, pgp_id:, trait_efo:])    [100%] 1 of 1 ✔
[86/c7ad10] process > PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)                                         [100%] 1 of 1 ✔
[skipped  ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM (BiomeCnics chromosome ALL)              [100%] 1 of 1, stored: 1 ✔
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR                                         -
[-        ] process > PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF                                                 -
[skipped  ] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1)                                      [100%] 1 of 1, stored: 1 ✔
[6e/3b81af] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (BiomeCnics chromosome ALL)            [100%] 1 of 1 ✔
[e8/a8fde7] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS (BiomeCnics GRCh37)                       [100%] 1 of 1 ✔
[59/188612] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF (reference)                            [100%] 1 of 1 ✔
[c9/3e550c] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED (BiomeCnics)                            [100%] 1 of 1 ✔
[da/ad229e] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS (BiomeCnics null pvar)                        [100%] 1 of 1 ✔
[02/04baf1] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET (BiomeCnics)                        [100%] 1 of 1 ✔
[55/d7b7f1] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT (BiomeCnics)                                [100%] 1 of 1 ✔
[ab/ccfa15] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA (reference)                                   [100%] 1 of 1 ✔
[7a/8d43dd] process > PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT (BiomeCnics)                              [100%] 1 of 1 ✔
[2c/f17610] process > PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (BiomeCnics chromosome ALL)                           [100%] 1 of 1 ✔
[5c/4b4538] process > PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE (BiomeCnics)                                           [100%] 1 of 1 ✔
[06/03fcf7] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES (BiomeCnics additive scorefile)             [100%] 1 of 1 ✔
[df/399166] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ (BiomeCnics null afreq)                          [100%] 1 of 1 ✔
[68/ff1f63] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE (reference chromosome ALL effect type additive 0) [100%] 2 of 2 ✔
[cf/ed0399] process > PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE (BiomeCnics)                                   [100%] 1 of 1 ✔
[1e/4f46f1] process > PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS (1)                                               [100%] 1 of 1 ✔
[b2/e0aea2] process > PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT (BiomeCnics)                                           [100%] 1 of 1, failed: 1 ✘
[58/3b133d] process > PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS (1)                                                   [100%] 1 of 1 ✔
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT (BiomeCnics)'

Caused by:
  Process `PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT (BiomeCnics)` terminated with an error exit status (1)

Command executed:

  export TMPDIR=$PWD # tmpdir must always be writable for quarto
  echo nextflow run pgscatalog/pgsc_calc --max_cpus 1 -profile singularity --input biome_cnics_plink/samplesheet.BiomeCNICS.csv --pgs_id PGS003446,PGS002237 --target_build GRCh37 -w /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work/ --genotypes_cache /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/test_nextflow_1/genotypes_cache --min_overlap 0.20 --run_ancestry /sc/arion/projects/CranioProject/HIV/HIV_SamreenPaola/PRS/pgsc-calc/resources/pgsc_HGDP+1kGP_v1.tar.zst -resume -latest > command.txt

  echo "keep_multiallelic: false" > params.txt
  echo "keep_ambiguous   : false"    >> params.txt
  echo "min_overlap      : 0.2"       >> params.txt

  quarto render report.qmd -M "self-contained:true"         -P score_path:BiomeCnics_pgs.txt.gz         -P sampleset:BiomeCnics         -P run_ancestry:true         -P reference_panel_name:HGDP+1kGP         -P version:2.0.0-beta         -o report.html

  cat <<-END_VERSIONS > versions.yml
  SCORE_REPORT:
      R: $(echo $(R --version 2>&1) | head -n 1 | cut -f 3 -d ' ')
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  error: Uncaught PermissionDenied: Permission denied (os error 13): mkdir '/tmp/quarto_cache_home/quarto'
          Deno.mkdirSync(dir, {
               ^
      at Object.mkdirSync (ext:deno_fs/30_fs.js:132:7)
      at ensureDirSync (file:///opt/quarto/bin/quarto.js:9288:14)
      at quartoDir (file:///opt/quarto/bin/quarto.js:33006:5)
      at quartoCacheDir (file:///opt/quarto/bin/quarto.js:32998:12)
      at file:///opt/quarto/bin/quarto.js:39501:25

Work dir:
  /hpc/users/zafers02/HIV_SamreenPaola/PRS/test_nextflow_1/work/b2/e0aea27e406e462a2f023c2720b3ba

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details
ERROR ~ ERROR: No results report written!

 -- Check '.nextflow.log' file for details
samreenzafer commented 4 months ago

I even tried the following and but got the same error. 


[zafers02@lc02a30 test_nextflow_1]$ echo $HOME
/hpc/users/zafers02
[zafers02@lc02a30 test_nextflow_1]$ mkdir -p $HOME/.cache/deno/gen
[zafers02@lc02a30 test_nextflow_1]$ mkdir -p $HOME/.cache/quarto
[zafers02@lc02a30 test_nextflow_1]$ ll  $HOME/.cache 
drwxrwxrwx 3 zafers02 nicolp01a   25 Jun 20 19:35 deno
drwxr-xr-x 2 zafers02 nicolp01a 4096 May 28 20:40 fontconfig
drwxrwxrwx 3 zafers02 nicolp01a  147 Sep 10  2021 matplotlib
drwxr-xr-x 3 zafers02 nicolp01a   61 Jun 14  2021 Python-Eggs
drwxrwxrwx  2 zafers02 nicolp01a   10 Jun 20 19:35 quarto
drwxrwxrwx 2 zafers02 nicolp01a   37 Jul 10  2023 rstudio
nebfield commented 4 months ago

This looks like a problem with the way singularity is configured on your cluster. /tmp/ should be writable in a singularity container.

Perhaps the conda profile might work better for you, or maybe your cluster admins can help with singularity configuration.

samreenzafer commented 4 months ago

Hi Benjamin, Our hpc recommended that I add the following code in the file ~/.nextflow/assets/pgscatalog/pgsc_calc/tests/config/nextflow.config

singularity {
        runOptions = "--bind \${SINGULARITY_TMPDIR}:[user_directory]"
}

And this worked for me to run my sample set with ancestry error free. I will now try to see if running PGS on each indivudal sample cohort will give me a better variant overlap, rather than this mergedCohort that I have. Thanks again for your help.