Closed Sungryong-Oh closed 2 months ago
Hi @Sungryong-Oh,
Could you post your full unifier command line please? i.e. the command with all the parameters you're using to run the unifier.
Thanks, Chris
Here is the full command line of Unify run:
/bin/bash /home/ohs5/monorail-external/singularity/run_recount_unify.sh /home/ohs5/monorail-external/singularity_image/recount-unify_1.1.2rc.sif(or recount-unify-1.1.1.sif) hg38 /home/ohs5/monorail-external/GENOME/ /home/ohs5/monorail-external/output/unify /home/ohs5/monorail-external/output/pump/output/ /home/ohs5/monorail-external/output/metadata.tsv 10 sra:208
Thanks, that looks fine.
Going back to error reported in your original post:
/tmp/) does not exist at /usr/bin/parallel line 3993```
The monorail unifier makes heavy use of the GNU Parallel program which appears to be the source of this error.
It's been quite a while since I ran this under Singularity and I don't usually test updates under it these days since my work has mostly been focused on Docker use cases.
Does the `/scr1/users/ohs5` path look familiar to you and do you have permissions to write there?
Dear Chris, Thanks for confirm.
Actually, I recently tried again and Unify worked well when I gave temporary directory only as /tmp. I realized that this is a kind of issue with "GNU Parallel". For some reason, GNU Parallel only works when I set TMPDIR as /tmp, not any other location. I set up TMPDIR under my personal one because I'm using it on hpc, but setting TMPDIR to other location seems not working at all.
Anyway, I don't know how to set the TMPDIR of GNUParallel when I'm using Unify, but it currently works and I successfully get an output. Please let me know if you can give me to change the setting of Parallel.
Thanks for your help.
thanks for the update, feel free to (re)open or open a new ticket if you continue to run into issues
Hi, I recently start use monorail pipeline again, but fail to make a unify output. even though pump pipeline seems like to be successfully finished.
I pulled the most recent version of monorail-pump v.1.3. I tested two unify version -v.1.1.2rc and v.1.1 but both fail to finish successfully. I ran it with singularity image.
Here is my error log:
[Mon Aug 19 23:44:20 2024] rule split_final_rejoined_exons: input: all.exon_counts.rejoined.tsv.gz, all.exon_counts.rejoined.tsv.gz.accession_header output: exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.G026.gz, exon_sums_per_study/08/SRP 433808/sra.exon_sums.SRP433808.G029.gz, exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.R109.gz , exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.F006.gz, exon_sums_per_study/08/SRP433808/sra .exon_sums.SRP433808.ERCC.gz, exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.SIRV.gz jobid: 19 threads: 10
ERCC,SIRV 1709834 /container-mounts/ref/exon_bitmasks.tsv /container-mounts/ref/exon_bitmask_coords.tsv all.exon_counts.rejoined.tsv.gz 10 rm -rf exons_split_by_study_temp exon_annotation_split_runs
Error in tempfile() using template /scr1/users/ohs5/tmp/parXXXXX.par: Parent directory (/scr1/users/ohs5 /tmp/) does not exist at /usr/bin/parallel line 3993. [Mon Aug 19 23:44:20 2024] Error in rule split_final_rejoined_exons: jobid: 19 output: exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.G026.gz, exon_sums_per_study/08/SRP 433808/sra.exon_sums.SRP433808.G029.gz, exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.R109.gz , exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.F006.gz, exon_sums_per_study/08/SRP433808/sra .exon_sums.SRP433808.ERCC.gz, exon_sums_per_study/08/SRP433808/sra.exon_sums.SRP433808.SIRV.gz
RuleException: CalledProcessError in line 333 of /recount-unify/Snakefile: Command ' set -euo pipefail;
/bin/bash /recount-unify/rejoin/split_out_exon_sums_by_study.sh sra G026,G029,R109,F006, ERCC,SIRV 1709834 /container-mounts/ref/exon_bitmasks.tsv /container-mounts/ref/exon_bitmask_coords.tsv all.exon_counts.rejoined.tsv.gz 10 rm -rf exons_split_by_study_temp exon_annotation_split_runs ' returned non-zero exit status 2. File "/recount-unify/Snakefile", line 333, in __rule_split_final_rejoined_exons File "/opt/conda/envs/recount-unify/lib/python3.9/concurrent/futures/thread.py", line 52, in run Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message
Hopefully to get an idea from you to fix it. Thanks!