Open Fiwx opened 3 weeks ago
I reran pgsc_calc with --resume after a loop device error and got this:
linux@b3d36862d500:~/urunner/system/test$ nextflow run /home/linux/urunner/tools/pgsc_calc/main.nf -profile singularity --input /home/linux/urunner/system/test/u10000_file8267_yofb_uk_s_uk.23andme_output/samplesheet.csv --scorefile '/home/linux/urunner/system/test/custom_scores/*.txt.gz' --target_build GRCh37 --min_overlap 0.0 --pca_maf_target 0.0 --run_ancestry /home/linux/urunner/data/pgsc_1000G_v1.tar.zst -c /home/linux/urunner/references/custom.config --genotypes_cache /home/linux/urunner/system/test/cache --resume
N E X T F L O W ~ version 24.04.4
Launching `/home/linux/urunner/tools/pgsc_calc/main.nf` [determined_mayer] DSL2 - revision: a10e219a42
------------------------------------------------------
pgscatalog/pgsc_calc v2.0.0-beta.3
------------------------------------------------------
Core Nextflow options
runName : determined_mayer
containerEngine: singularity
launchDir : /home/linux/urunner/system/test
workDir : /home/linux/urunner/system/test/work projectDir : /home/linux/urunner/tools/pgsc_calc userName : linux
profile : singularity configFiles :
!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------If you use pgscatalog/pgsc_calc for your analysis please cite:
* The Polygenic Score Catalog
https://doi.org/10.1101/2024.05.29.24307783
https://doi.org/10.1038/s41588-021-00783-5
* The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
* Software dependencies
https://github.com/pgscatalog/pgsc_calc/blob/main/CITATIONS.md
Reference database provided: skipping bootstrap
executor > local (3)
[dd/84dcdb] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1) [100%] 1 of 1 ✔
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR -
[skipped ] PGS…CALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1, stored: 1 ✔
[skipped ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1) [100%] 1 of 1, stored: 1 ✔
[cd/924b37] PGS…CALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (u10000file8267yofbuksuk chromosome ALL) [ 0%] 0 of 1
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT -
[db/237c48] PGS…G_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (u10000file8267yofbuksuk chromosome ALL) [ 0%] 0 of 1
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_AFREQ -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:PLINK2_SCORE -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:SCORE_AGGREGATE -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:ANCESTRY_ANALYSIS -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:REPORT:SCORE_REPORT -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:DUMPSOFTWAREVERSIONS -
ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (u10000file8267yofbuksuk chromosome ALL)'
Caused by:
Missing output file(s) `u10000file8267yofbuksuk_ALL_matched.txt.gz` expected by process `PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (u10000file8267yofbuksuk chromosome ALL)`
Command executed:
pgscatalog-intersect --ref GRCh37_1000G_ALL.pvar.zst --target GRCh37_u10000file8267yofbuksuk_ALL.pvar.zst --chrom ALL --maf_target 0.0 --geno_miss 0.1 --outdir . -v
n_matched=$(sed -n '3p' intersect_counts_ALL.txt)
if [ $n_matched == "0" ]
then
echo "ERROR: No variants in intersection"
exit 1
else
mv matched_variants.txt.gz u10000file8267yofbuksuk_ALL_matched.txt.gz
fi
cat <<-END_VERSIONS > versions.yml
INTERSECT_VARIANTS:
pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.__version__)'))
END_VERSIONS
Command exit status:
0
Command output:
(empty)
Command error:
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:12 INFO Processed 6000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:18 INFO Processed 6500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:24 INFO Processed 7000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:30 INFO Processed 7500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:36 INFO Processed 8000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:42 INFO Processed 8500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:48 INFO Processed 9000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:54 INFO Processed 9500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:40:00 INFO Processed 10000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:40:06 INFO Processed 10500000
[...]
TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:41:08 INFO Processed 15500000 TARGET variants
executor > local (3)
[dd/84dcdb] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1) [100%] 1 of 1 ✔
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR -
[skipped ] PGS…CALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1, stored: 1 ✔
[skipped ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1) [100%] 1 of 1, stored: 1 ✔
[cd/924b37] PGS…CALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1, failed: 1 ✘
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS -
[- ]
[...]
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:30 INFO Processed 7500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:36 INFO Processed 8000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:42 INFO Processed 8500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:48 INFO Processed 9000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:39:54 INFO Processed 9500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:40:00 INFO Processed 10000000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:40:06 INFO Processed 10500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:40:12 INFO Processed 11000000
[...]
TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:41:08 INFO Processed 15500000 TARGET variants
executor > local (3)
[dd/84dcdb] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1) [100%] 1 of 1 ✔
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR -
[skipped ] PGS…CALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1, stored: 1 ✔
[skipped ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1) [100%] 1 of 1, stored: 1 ✔
[cd/924b37] PGS…CALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1, failed: 1 ✘
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT -
[db/237c48] PGS…G_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1 ✔
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:APPLY_SCORE:RELABEL_SCOREFILES
[...]
TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:41:08 INFO Processed 15500000 TARGET variants
executor > local (3)
[dd/84dcdb] PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1) [100%] 1 of 1 ✔
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELBIM -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_RELABELPVAR -
[skipped ] PGS…CALC:PGSCCALC:MAKE_COMPATIBLE:PLINK2_VCF (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1, stored: 1 ✔
[skipped ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:EXTRACT_DATABASE (1) [100%] 1 of 1, stored: 1 ✔
[cd/924b37] PGS…CALC:ANCESTRY_PROJECT:INTERSECT_VARIANTS (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1, failed: 1 ✘
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FILTER_VARIANTS -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_REF -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:INTERSECT_THINNED -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:RELABEL_IDS -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_MAKEBED_TARGET -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:PLINK2_ORIENT -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PCA -
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:ANCESTRY_PROJECT:FRAPOSA_PROJECT -
[db/237c48] PGS…G_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (u10000file8267yofbuksuk chromosome ALL) [100%] 1 of 1 ✔
[- ] PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_COMBINE -
[...]
TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:43:40 INFO Processed 27904794 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:43:40 INFO Outputting TARGET variants -> target_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:45:05 INFO Joining & outputting matched variants -> matched_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO 27904794/27904794 (100.00%) of TARGET variants matched the REFERENCE data
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO 23599799/27904794 (84.57%) of matched variants are eligible for PCA
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO Outputting variant counts -> intersect_counts_ALL.txt
Work dir:
/home/linux/urunner/system/test/work/cd/924b37caaa9eb2d2c92578efb38cfd
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
-- Check '.nextflow.log' file for details
.command.out is empty.
.command.err ends in this:
[...]
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:43:40 INFO Processed 27904794 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:43:40 INFO Outputting TARGET variants -> target_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:45:05 INFO Joining & outputting matched variants -> matched_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO 27904794/27904794 (100.00%) of TARGET variants matched the REFERENCE data
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO 23599799/27904794 (84.57%) of matched variants are eligible for PCA
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO Outputting variant counts -> intersect_counts_ALL.txt
.command.log ends in this:
[...]
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:43:40 INFO Outputting TARGET variants -> target_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:45:05 INFO Joining & outputting matched variants -> matched_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO 27904794/27904794 (100.00%) of TARGET variants matched the REFERENCE data
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO 23599799/27904794 (84.57%) of matched variants are eligible for PCA
pgscatalog.match.cli.intersect_cli: 2024-08-25 17:51:18 INFO Outputting variant counts -> intersect_counts_ALL.txt
cp: '.command.out' and '/home/linux/urunner/system/test/work/cd/924b37caaa9eb2d2c92578efb38cfd/.command.out' are the same file
cp: '.command.err' and '/home/linux/urunner/system/test/work/cd/924b37caaa9eb2d2c92578efb38cfd/.command.err' are the same file
cp: '.command.trace' and '/home/linux/urunner/system/test/work/cd/924b37caaa9eb2d2c92578efb38cfd/.command.trace' are thesame file
mkdir: cannot create directory '/home/linux/urunner/system/test/cache/ancestry/intersected': Permission denied
mv: cannot move 'intersect_counts_ALL.txt' to '/home/linux/urunner/system/test/cache/ancestry/intersected/.': No such file or directory
mkdir: cannot create directory '/home/linux/urunner/system/test/cache/ancestry/intersected': Permission denied
mv: cannot move 'user10000file8266yearofbirthunknownsexunknown_ALL_matched.txt.gz' to '/home/linux/urunner/system/test/cache/ancestry/intersected/.': No such file or directory
mkdir: cannot create directory '/home/linux/urunner/system/test/cache/ancestry/intersected': Permission denied
mv: cannot move 'versions.yml' to '/home/linux/urunner/system/test/cache/ancestry/intersected/.': No such file or directory
.command.sh:
#!/bin/bash -euo pipefail
pgscatalog-intersect --ref GRCh37_1000G_ALL.pvar.zst --target GRCh37_user10000file8266yearofbirthunknownsexunknown_ALL.pvar.zst --chrom ALL --maf_target 0.0 --geno_miss 0.1 --outdir . -v
n_matched=$(sed -n '3p' intersect_counts_ALL.txt)
if [ $n_matched == "0" ]
then
echo "ERROR: No variants in intersection"
exit 1
else
mv matched_variants.txt.gz user10000file8266yearofbirthunknownsexunknown_ALL_matched.txt.gz
fi
cat <<-END_VERSIONS > versions.yml
INTERSECT_VARIANTS:
pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.__version__)'))
END_VERSIONS
.command.trace:
nextflow.trace/v2
realtime=1854040
%cpu=1018
cpu_model=Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
rchar=7668009450
wchar=6997428096
syscr=704235
syscw=698891
read_bytes=555186176
write_bytes=6997999616
%mem=5
vmem=608252
rss=316464
peak_vmem=608252
peak_rss=316464
vol_ctxt=62315
inv_ctxt=2864
I ran the exact command that was in .command.sh, and it succeeded without an error:
linux@b3d36862d500:~/urunner/system/test/work/cd/924b37caaa9eb2d2c92578efb38cfd$ pgscatalog-intersect --ref GRCh37_1000G_ALL.pvar.zst --target GRCh37_user10000file8266yearofbirthunknownsexunknown_ALL.pvar.zst --chrom ALL --maf_target 0.0 --geno_miss 0.1 --outdir . -v
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:22:56 DEBUG Verbose logging enabled
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:22:56 INFO Reading REFERENCE variants: GRCh37_1000G_ALL.pvar.zst
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:23:01 INFO Processed 500000 REFERENCE variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:23:05 INFO Processed 1000000 REFERENCE variants
[...]
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:35:46 INFO Processed 84500000 REFERENCE variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:35:49 INFO Processed 84805772 REFERENCE variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:35:49 INFO Outputting REFERNCE variants -> reference_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:40:06 INFO Reading TARGET variants: GRCh37_user10000file8266yearofbirthunknownsexunknown_ALL.pvar.zst
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:40:12 INFO Processed 500000 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:40:19 INFO Processed 1000000 TARGET variants
[...]
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:45:35 INFO Processed 27904794 TARGET variants
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:45:35 INFO Outputting TARGET variants -> target_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:46:52 INFO Joining & outputting matched variants -> matched_variants.txt.gz
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:52:27 INFO 27904794/27904794 (100.00%) of TARGET variants matched the REFERENCE data
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:52:27 INFO 23599799/27904794 (84.57%) of matched variants are eligible for PCA
pgscatalog.match.cli.intersect_cli: 2024-08-25 18:52:27 INFO Outputting variant counts -> intersect_counts_ALL.txt
The original loop devices error happens with and without this recommendation: https://github.com/PGScatalog/pgsc_calc/issues/361#issuecomment-2296923470, and on both 64 and 32 GB of RAM, and also with either 65 or 93 scorefiles.
Hmmm, we might have to investigate this further. It could also be some interaction with how singularity was initially set up/administered. You could also try conda to see if that works?
I will look into conda. The loop devices themselves look normal. Perhaps it is some setup issue that occurs under certain conditions, since I previously used the same setup without seeing this error.
I can still replicate the error but I don't think it's the root cause of the pipeline failing to run:
Sep-12 19:41:35.981 [main] DEBUG nextflow.Session - Failed to invoke observer completion handler: nextflow.prov.ProvObserver@4a9e4be7
java.lang.NullPointerException: null
at nextflow.prov.BcoRenderer$_render_closure3.doCall(BcoRenderer.groovy:93)
at nextflow.prov.BcoRenderer$_render_closure3.call(BcoRenderer.groovy)
...
at nextflow.cli.Launcher.main(Launcher.groovy:657)
Does the problem still happen if you set:
to enabled = false
?
Description of the bug
I ran the current version of pgsc_calc with 63 scorefiles on a single sample and got this error:
I re-ran pgsc_calc with the --resume flag, and it seemed to work (correctly generated the _pgs.txt.gz file), but also it had this strange error:
It says there was an error when trying to render the BioCompute Object (BCO), as indicated by the NullPointerException in the BcoRenderer, which I haven't heard of.
Do you have any ideas about what could be causing this? It is odd that resuming with no other changes works, but it doesn't work on its own the first try. I replicated this on another machine with a different sample file.
Command used and terminal output
Relevant files
No response
System information
Nextflow version: 24.04.4 build 5917 System: Linux 6.5.0-1022-aws Runtime: Groovy 4.0.21 on OpenJDK 64-Bit Server VM 11.0.24+8-post-linux-1linux322.04 Encoding: UTF-8 (ANSI_X3.4-1968) CPUs: 8 - Mem: 62.1 GB (8.9 GB) - Swap: 0 (0)
"/home/linux/urunner/system/test/file10001_filestudy_yearofsample_unknown_sex_unknown.e_output/samplesheet.csv", "format": "csv", "scorefile": "/home/linux/urunner/system/test/custom_scores/*.txt.gz", "pgs_id": null, "trait_efo": null, "efo_id": null, "pgp_id": null, "efo_direct": false, "run_ancestry": "/home/linux/urunner/data/pgsc_1000G_v1.tar.zst", "ancestry_checksums": "/home/linux/urunner/tools/pgsc_calc/assets/ancestry/checksums.txt", "hg19_chain": null, "hg38_chain": null, "load_afreq": true, "geno_ref": 0.1, "mind_ref": 0.1, "maf_ref": 0.05, "hwe_ref": 0.0001, "indep_pairwise_ref": "1000 50 0.05", "ld_grch37": "/home/linux/urunner/tools/pgsc_calc/assets/ancestry/high-LD-regions-hg19-GRCh37.txt", "ld_grch38": "/home/linux/urunner/tools/pgsc_calc/assets/ancestry/high-LD-regions-hg38-GRCh38.txt", "ref_format_version": "v0.1", "ref_samplesheet": "/home/linux/urunner/tools/pgsc_calc/assets/ancestry/reference.csv", "projection_method": "oadp", "ancestry_method": "RandomForest", "ref_label": "SuperPop", "n_popcomp": 5, "normalization_method": "empirical mean mean+var", "n_normalization": 4, "pca_maf_target": 0.0, "pca_geno_miss_target": 0.1, "liftover": false, "target_build": "GRCh37", "min_lift": 0.95, "min_overlap": 0.0, "keep_ambiguous": false, "keep_multiallelic": false, "copy_genomes": false, "genotypes_cache": "/home/linux/urunner/system/test/cache", "only_bootstrap": false, "only_input": false, "only_compatible": false, "only_match": false, "only_projection": false, "only_score": false, "skip_ancestry": true, "outdir": "/home/linux/urunner/system/test/results", "publish_dir_mode": "copy", "email": null, "email_on_fail": null, "plaintext_email": false, "monochrome_logs": false, "hook_url": null, "help": false, "version": false, "config_profile_name": null, "config_profile_description": null, "custom_config_version": "master", "resume": true, "custom_config_base": "https://raw.githubusercontent.com/nf-core/configs/master", "config_profile_contact": null, "config_profile_url": null, "max_memory": "62.GB", "max_cpus": 8, "max_time": "240.h"