search

kundajelab / atac_dnase_pipelines

ATAC-seq and DNase-seq processing pipeline
BSD 3-Clause "New" or "Revised" License
159 stars 81 forks source link

shuf: read error: Cannot allocate memory at spr_tag_PE step #121

Closed chesi closed 6 years ago

chesi commented 6 years ago

Hi, how do I increase the memory allocation for this step: spr_tag_PE? The default 12G is not enough. This step is not listed in the atac.config file. I attach the bds.log file. Thanks! bds.log

leepc12 commented 6 years ago

Use -mem_shuf 20G.

Jin

On Mon, Jun 4, 2018 at 7:12 AM, Alessandra notifications@github.com wrote:

Hi, how do I increase the memory allocation for this step: spr_tag_PE? The default 12G is not enough. This step is not listed in the atac.config file. I attach the bds.log file. Thanks! bds.log https://github.com/kundajelab/atac_dnase_pipelines/files/2068571/bds.log

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kundajelab/atac_dnase_pipelines/issues/121, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOd_J8PS6bYgNdjfX-pjAAhCvlJRa3vks5t5UBSgaJpZM4UZKc9 .

chesi commented 6 years ago

Thanks!

Alessandra

From: Jin Lee notifications@github.com Reply-To: kundajelab/atac_dnase_pipelines reply@reply.github.com Date: Monday, June 4, 2018 at 12:25 PM To: kundajelab/atac_dnase_pipelines atac_dnase_pipelines@noreply.github.com Cc: Alessandra a.chesi@gmail.com, Author author@noreply.github.com Subject: Re: [kundajelab/atac_dnase_pipelines] shuf: read error: Cannot allocate memory at spr_tag_PE step (#121)

Use -mem_shuf 20G.

Jin

On Mon, Jun 4, 2018 at 7:12 AM, Alessandra notifications@github.com wrote:

Hi, how do I increase the memory allocation for this step: spr_tag_PE? The default 12G is not enough. This step is not listed in the atac.config file. I attach the bds.log file. Thanks! bds.log https://github.com/kundajelab/atac_dnase_pipelines/files/2068571/bds.log

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kundajelab/atac_dnase_pipelines/issues/121, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOd_J8PS6bYgNdjfX-pjAAhCvlJRa3vks5t5UBSgaJpZM4UZKc9 .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

chesi commented 6 years ago

Thanks!

Also, is there a rule of thumb to estimate the memory requirements for the atacqc step?

My jobs keep dying because they do not have enough memory, and I am increasing allocation in steps, but it takes a long time to guess the right amount of memory.

Thanks,

A.

From: Jin Lee notifications@github.com Reply-To: kundajelab/atac_dnase_pipelines reply@reply.github.com Date: Monday, June 4, 2018 at 12:25 PM To: kundajelab/atac_dnase_pipelines atac_dnase_pipelines@noreply.github.com Cc: Alessandra a.chesi@gmail.com, Author author@noreply.github.com Subject: Re: [kundajelab/atac_dnase_pipelines] shuf: read error: Cannot allocate memory at spr_tag_PE step (#121)

Use -mem_shuf 20G.

Jin

On Mon, Jun 4, 2018 at 7:12 AM, Alessandra notifications@github.com wrote:

Hi, how do I increase the memory allocation for this step: spr_tag_PE? The default 12G is not enough. This step is not listed in the atac.config file. I attach the bds.log file. Thanks! bds.log https://github.com/kundajelab/atac_dnase_pipelines/files/2068571/bds.log

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kundajelab/atac_dnase_pipelines/issues/121, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOd_J8PS6bYgNdjfX-pjAAhCvlJRa3vks5t5UBSgaJpZM4UZKc9 .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

leepc12 commented 6 years ago

Sorry, it's hard to estimate. Please start with large enough memory...