A simple test run was successful.
But, the real Pore-C data (Brahman cow liver sample) was not.
At the last stage, it causes an issue.
[Mon Aug 8 01:43:13 2022]
Error in rule create_hic:
jobid: 5
output: /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic
log: /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic.log (check log file(s) for error message)
conda-env: /home/uqhjung3/miniconda3/envs/Pore-C-Snakemake/.snakemake/conda/1c2800ef
shell:
java -Xmx2g -jar /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/juicer_tools.jar pre /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/pairs/NlaIII_run01_BrahChr_unphased.sorted.pairs.gz /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/refgenome/BrahChr.rg.chromsizes -f /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_BrahChr.hicRef &>/scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic.log
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Removing output files of failed job create_hic since they might be corrupted:
/scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic
[Tue Aug 9 01:43:41 2022]
Finished job 6.
7006 of 7008 steps (100%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/uqhjung3/miniconda3/envs/Pore-C-Snakemake/.snakemake/log/2022-08-02T211044.528701.snakemake.log
########################### Job Execution History #############################
JobId:258567.flashmgr2
UserName:uqhjung3
GroupName:qris-uq
JobName:BrahM_PrCT1
SessionId:3154
ResourcesRequested:mem=450gb,ncpus=20,place=free,walltime=333:00:00
ResourcesUsed:cpupercent=1977,cput=2036:53:59,mem=129800580kb,ncpus=20,vmem=183438556kb,walltime=148:34:45
QueueUsed:General
AccountString:qris-uq
ExitStatus:1
###############################################################################
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARN [2022-08-08T00:30:19,474] [Globals.java:138] [main] Development mode is enabled
Start preprocess
Writing header
Writing body
..........................................................................................................................................................................................Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.regex.Pattern.matcher(Pattern.java:1133)
at java.base/java.lang.String.replaceAll(String.java:2126)
at juicebox.data.ChromosomeHandler.cleanUpName(ChromosomeHandler.java:77)
at juicebox.tools.utils.original.AsciiPairIterator.advance(AsciiPairIterator.java:177)
at juicebox.tools.utils.original.AsciiPairIterator.next(AsciiPairIterator.java:268)
at juicebox.tools.utils.original.Preprocessor.writeBody(Preprocessor.java:689)
at juicebox.tools.utils.original.Preprocessor.preprocess(Preprocessor.java:419)
at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:122)
at juicebox.tools.HiCTools.main(HiCTools.java:94)
After googling and GitHub issues (3D Genomics and Pore-C), a bit of conflicting info was provided.
Using a keyword search “sun.reflect.Reflection.getCallerClass”, I could not find anything from Pore-C GitHub. However, from 3D Genomics, I found this.
Questions;
Q1: Is there any effective way to fix this matter?
Q2: Is there any option that I can resubmit (resume) the job to start from the interrupted stage? It has been running for a week.
Hi all,
Hope this email finds you well. I have questions for you about the Pore-C pipeline.
The program was installed according to these links.
https://github.com/nanoporetech/pore-c https://github.com/nanoporetech/Pore-C-Snakemake
A simple test run was successful. But, the real Pore-C data (Brahman cow liver sample) was not. At the last stage, it causes an issue.
[Mon Aug 8 01:43:13 2022] Error in rule create_hic: jobid: 5 output: /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic log: /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic.log (check log file(s) for error message) conda-env: /home/uqhjung3/miniconda3/envs/Pore-C-Snakemake/.snakemake/conda/1c2800ef shell: java -Xmx2g -jar /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/juicer_tools.jar pre /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/pairs/NlaIII_run01_BrahChr_unphased.sorted.pairs.gz /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/refgenome/BrahChr.rg.chromsizes -f /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_BrahChr.hicRef &>/scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic.log (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Removing output files of failed job create_hic since they might be corrupted: /scratch/project/multi-contact-pore-c/BrahMom_PoreC1/resultsT1.test/juicebox/NlaIII_run01_BrahChr_unphased.hic [Tue Aug 9 01:43:41 2022] Finished job 6. 7006 of 7008 steps (100%) done Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /home/uqhjung3/miniconda3/envs/Pore-C-Snakemake/.snakemake/log/2022-08-02T211044.528701.snakemake.log ########################### Job Execution History ############################# JobId:258567.flashmgr2 UserName:uqhjung3 GroupName:qris-uq JobName:BrahM_PrCT1 SessionId:3154 ResourcesRequested:mem=450gb,ncpus=20,place=free,walltime=333:00:00 ResourcesUsed:cpupercent=1977,cput=2036:53:59,mem=129800580kb,ncpus=20,vmem=183438556kb,walltime=148:34:45 QueueUsed:General AccountString:qris-uq ExitStatus:1 ###############################################################################
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARN [2022-08-08T00:30:19,474] [Globals.java:138] [main] Development mode is enabled Start preprocess Writing header Writing body ..........................................................................................................................................................................................Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.base/java.util.regex.Pattern.matcher(Pattern.java:1133) at java.base/java.lang.String.replaceAll(String.java:2126) at juicebox.data.ChromosomeHandler.cleanUpName(ChromosomeHandler.java:77) at juicebox.tools.utils.original.AsciiPairIterator.advance(AsciiPairIterator.java:177) at juicebox.tools.utils.original.AsciiPairIterator.next(AsciiPairIterator.java:268) at juicebox.tools.utils.original.Preprocessor.writeBody(Preprocessor.java:689) at juicebox.tools.utils.original.Preprocessor.preprocess(Preprocessor.java:419) at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:122) at juicebox.tools.HiCTools.main(HiCTools.java:94)
After googling and GitHub issues (3D Genomics and Pore-C), a bit of conflicting info was provided. Using a keyword search “sun.reflect.Reflection.getCallerClass”, I could not find anything from Pore-C GitHub. However, from 3D Genomics, I found this.
https://groups.google.com/g/3d-genomics/search?q=sun.reflect.Reflection.getCallerClass
1: Java version/lang issue
Questions; Q1: Is there any effective way to fix this matter? Q2: Is there any option that I can resubmit (resume) the job to start from the interrupted stage? It has been running for a week.
Cheers,
Taek