search

nf-core / hicar

Pipeline for HiCAR data, a robust and sensitive multi-omic co-assay for simultaneous measurement of transcriptome, chromatin accessibility and cis-regulatory chromatin contacts.
https://nf-co.re/hicar
MIT License
5 stars 5 forks source link

Java heap space #108

Open luciazifcakova opened 2 months ago

luciazifcakova commented 2 months ago

Description of the bug

I get "Java heap space" error for any configuration of Nextflow versions even though I have added "NXF_OPTS='-Xms1g -Xmx4g'" to my .bashrc...

N E X T F L O W ~ version 22.04.5 Launching https://github.com/nf-core/hicar [voluminous_noyce] DSL2 - revision: 429087d2b1 [1.0.0]------------------------------------------------------ ,--./,-. _ _ /,-..--~' |\ | | / ` / \ |) | } { | | | _, __/ | \ |__ `-.,--, .,.,' nf-core/hicar v1.0.0

Core Nextflow options revision : 1.0.0 runName : voluminous_noyce containerEngine : singularity launchDir : /flash/MillerU/S_lessoniana_hic_maps workDir : /flash/MillerU/S_lessoniana_hic_maps/work projectDir : /home/l/lucia-zifcakova/.nextflow/assets/nf-core/hicar userName : lucia-zifcakova profile : oist, configFiles : /home/l/lucia-zifcakova/.nextflow/assets/nf-core/hicar/nextflow.configInput/output options input : samples.csv outdir : ./Reference genome options fasta : /bucket/MillerU/Zifcakova/Dovetail_ika_genomes/Dovetail_annotated_v_3_01_2022/S_les_scaffolds_soft_masked/S_les_scaffolds.fa bwa_index : /bucket/MillerU/Zifcakova/Dovetail_ika_genomes/Dovetail_annotated_v_3_01_2022/S_les_scaffolds_soft_masked/S_les_scaffolds.bwt gff : /bucket/MillerU/Zifcakova/s_les_tomo_anot/tsebra4_renamed_2.gffMAPS peak calling options maps_digest_file : null peak_pair_block : 1E+8Options related to tracks, juicer_tools, and circos juicer_tools_jar : https://github.com/aidenlab/Juicebox/releases/download/v2.13.06/juicer_tools_2.13.06.jarOther options not expose enrichment_fdr : 0.05Institutional config options config_profile_description: The Okinawa Institute of Science and Technology Graduate University (OIST) HPC cluster profile provided by nf-core/configs. config_profile_contact : OISTs Bioinformatics User Group [BioinfoUgrp@oist.jp](mailto:BioinfoUgrp@oist.jp) config_profile_url : https://github.com/nf-core/configs/blob/master/docs/oist.mdMax job request options max_cpus : 128 max_memory : 500 GB max_time : 3d 18h!! Only displaying parameters that differ from the pipeline defaults !!

If you use nf-core/hicar for your analysis please cite: The pipeline https://doi.org/10.5281/zenodo.5618247 The nf-core framework https://doi.org/10.1038/s41587-020-0439-x* Software dependencies https://github.com/nf-core/hicar/blob/master/CITATIONS.md

[- ] process > NFCORE_HICAR:HICAR:INPUT_CHECK:SAMPLESHEET_CHECK - [- ] process > NFCORE_HICAR:HICAR:CHECKSUMS - [- ] process > NFCORE_HICAR:HICAR:PREPARE_GENOME:GFFREAD - [- ] process > NFCORE_HICAR:HICAR:PREPARE_GENOME:GTF2BED - [- ] process > NFCORE_HICAR:HICAR:PREPARE_GENOME:CHROMSIZES - Java heap space -- Check script '/home/l/lucia-zifcakova/.nextflow/assets/nf-core/hicar/./workflows/../subworkflows/local/preparegenome.nf' at line: 85 or see '.nextflow.log' file for more details

Command used and terminal output

#!/bin/bash
#SBATCH --job-name=S_less_hicar
#SBATCH --partition=compute
#SBATCH --time=3-0
#SBATCH --mem=250G
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=20
#SBATCH --mail-user=lucia.zifcakova@oist.jp
#SBATCH --mail-type=BEGIN,FAIL,END
#SBATCH --output=./S_less_hicar.out

ml bioinfo-ugrp-modules
ml DebianMed
ml singularity
ml nf-core

nextflow run nf-core/hicar -r dev --input samples.csv --outdir ./ --gff /bucket/MillerU/Zifcakova/s_les_tomo_anot/tsebra4_renamed_2.gff -profile oist, singularity --fasta /bucket/MillerU/Zifcakova/Dovetail_ika_genomes/Dovetail_annotated_v_3_01_2022/S_les_scaffolds_soft_masked/S_les_scaffolds.fa --bwa_index /bucket/MillerU/Zifcakova/Dovetail_ika_genomes/Dovetail_annotated_v_3_01_2022/S_les_scaffolds_soft_masked/S_les_scaffolds --skip_diff_analysis

Relevant files

No response

System information

Currently Loaded Modules: 1) bioinfo-ugrp-modules 2) DebianMed/12.0 3) singularity/3.5.2 4) java-jdk/14 5) Nextflow2/22.04.5 6) nf-core/2.13.1 HPC, CentOS 8.X with the 4.XX version of the Linux kernel

jianhong commented 2 months ago

Try to add macs_gsize to your parameter list to avoid loading huge genome into the memory. And there is a typo in your script at -profile oist, singularity. Should be -profile oist,singularity

luciazifcakova commented 2 months ago

Dear Jianhong,

thank you for the information, it helped. I actually have Omni-C data that were cut by DNAse I but it is not on the list of enzymes. Can I add my own enzyme to the pipeline somehow?

Best wishes, Lucia

From: JIANHONG OU @.> Sent: Monday, April 22, 2024 10:40 PM To: nf-core/hicar @.> Cc: Lucia Zifcakova @.>; Author @.> Subject: Re: [nf-core/hicar] Java heap space (Issue #108)

Try to add macs_gsize to your parameter list to avoid loading huge genome into the memory. And there is a typo in your script at -profile oist, singularity. Should be -profile oist,singularity

— Reply to this email directly, view it on GitHubhttps://github.com/nf-core/hicar/issues/108#issuecomment-2069476077, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWQLEWBNMHLFOGUYAX2P57LY6UHLTAVCNFSM6AAAAABF757ORSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRZGQ3TMMBXG4. You are receiving this because you authored the thread.Message ID: @.***>

jianhong commented 2 months ago

Could you please let me know the enzyme? I can add it. My apologies that I did not write full documentation for that.Best!Your sincerely,Jianhong OuOn Apr 26, 2024, at 5:05 AM, luciazifcakova @.***> wrote: Dear Jianhong,

thank you for the information, it helped.

I actually have Omni-C data that were cut by DNAse I but it is not on the list of enzymes. Can I add my own enzyme to the pipeline somehow?

Best wishes,

Lucia

From: JIANHONG OU @.***>

Sent: Monday, April 22, 2024 10:40 PM

To: nf-core/hicar @.***>

Cc: Lucia Zifcakova @.>; Author @.>

Subject: Re: [nf-core/hicar] Java heap space (Issue #108)

Try to add macs_gsize to your parameter list to avoid loading huge genome into the memory. And there is a typo in your script at -profile oist, singularity. Should be -profile oist,singularity

—

Reply to this email directly, view it on GitHubhttps://github.com/nf-core/hicar/issues/108#issuecomment-2069476077, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWQLEWBNMHLFOGUYAX2P57LY6UHLTAVCNFSM6AAAAABF757ORSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRZGQ3TMMBXG4.

You are receiving this because you authored the thread.Message ID: @.***>

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

luciazifcakova commented 2 months ago

Dear Jianhong,

Omni-C use this enzyme: DNase I, a sequence-independent endonuclease. I don't know how can you implement this int he workflow...

Best wishes, Lucia

From: JIANHONG OU @.> Sent: Friday, April 26, 2024 6:50 PM To: nf-core/hicar @.> Cc: Lucia Zifcakova @.>; Author @.> Subject: Re: [nf-core/hicar] Java heap space (Issue #108)

Could you please let me know the enzyme? I can add it. My apologies that I did not write full documentation for that.Best!Your sincerely,Jianhong OuOn Apr 26, 2024, at 5:05 AM, luciazifcakova @.***> wrote: Dear Jianhong,

thank you for the information, it helped.

I actually have Omni-C data that were cut by DNAse I but it is not on the list of enzymes. Can I add my own enzyme to the pipeline somehow?

Best wishes,

Lucia

From: JIANHONG OU @.***>

Sent: Monday, April 22, 2024 10:40 PM

To: nf-core/hicar @.***>

Cc: Lucia Zifcakova @.>; Author @.>

Subject: Re: [nf-core/hicar] Java heap space (Issue #108)

Try to add macs_gsize to your parameter list to avoid loading huge genome into the memory. And there is a typo in your script at -profile oist, singularity. Should be -profile oist,singularity

—

Reply to this email directly, view it on GitHubhttps://github.com/nf-core/hicar/issues/108#issuecomment-2069476077, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWQLEWBNMHLFOGUYAX2P57LY6UHLTAVCNFSM6AAAAABF757ORSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRZGQ3TMMBXG4.

You are receiving this because you authored the thread.Message ID: @.***>

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://github.com/nf-core/hicar/issues/108#issuecomment-2079035955, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWQLEWE4WJNCOSIL3DSDMJDY7IPP3AVCNFSM6AAAAABF757ORSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZGAZTKOJVGU. You are receiving this because you authored the thread.Message ID: @.***>

jianhong commented 1 month ago

Sorry for the delay. Maybe hicar pipeline is not the best pipeline for you to analysis Omni-C. I can adjust the pipeline by adding skip parameter to skip the parse selection based on cutting site in the same fragment and also skip the cutting sites counting for interaction calling. However, I am afraid this change will take time.

Jianhong.

From: luciazifcakova @.> Date: Monday, April 29, 2024 at 7:16 PM To: nf-core/hicar @.> Cc: JIANHONG OU @.>, Comment @.> Subject: Re: [nf-core/hicar] Java heap space (Issue #108) Dear Jianhong,

Omni-C use this enzyme: DNase I, a sequence-independent endonuclease. I don't know how can you implement this int he workflow...

Best wishes, Lucia

From: JIANHONG OU @.> Sent: Friday, April 26, 2024 6:50 PM To: nf-core/hicar @.> Cc: Lucia Zifcakova @.>; Author @.> Subject: Re: [nf-core/hicar] Java heap space (Issue #108)

Could you please let me know the enzyme? I can add it. My apologies that I did not write full documentation for that.Best!Your sincerely,Jianhong OuOn Apr 26, 2024, at 5:05 AM, luciazifcakova @.***> wrote: Dear Jianhong,

thank you for the information, it helped.

I actually have Omni-C data that were cut by DNAse I but it is not on the list of enzymes. Can I add my own enzyme to the pipeline somehow?

Best wishes,

Lucia

From: JIANHONG OU @.***>

Sent: Monday, April 22, 2024 10:40 PM

To: nf-core/hicar @.***>

Cc: Lucia Zifcakova @.>; Author @.>

Subject: Re: [nf-core/hicar] Java heap space (Issue #108)

Try to add macs_gsize to your parameter list to avoid loading huge genome into the memory. And there is a typo in your script at -profile oist, singularity. Should be -profile oist,singularity

—

Reply to this email directly, view it on GitHubhttps://github.com/nf-core/hicar/issues/108#issuecomment-2069476077, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWQLEWBNMHLFOGUYAX2P57LY6UHLTAVCNFSM6AAAAABF757ORSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRZGQ3TMMBXG4.

You are receiving this because you authored the thread.Message ID: @.***>

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://github.com/nf-core/hicar/issues/108#issuecomment-2079035955, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWQLEWE4WJNCOSIL3DSDMJDY7IPP3AVCNFSM6AAAAABF757ORSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZGAZTKOJVGU. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://github.com/nf-core/hicar/issues/108#issuecomment-2083842569, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABLBEA46GLSMBPB63MYFAEDY73IELAVCNFSM6AAAAABF757ORSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBTHA2DENJWHE. You are receiving this because you commented.Message ID: @.***>