search

zstephens / neat-genreads

NEAT read simulation tools
Other
95 stars 27 forks source link

Getting memory error while using hg19 as reference and targeted bed file for whole exome #61

Open mrpk123 opened 5 years ago

mrpk123 commented 5 years ago

Hi,

I am trying to simulate variants for whole exome with a targeted bed file. As per your suggestion, i divided the job into 23 jobs. However, i am getting error as shown below. Same error was generated when i tried with 50 and 100 as number of jobs. Would you please help me in resolving this error? I am using hg19.fasta as reference.

My command is: python genReads.py -r hg19.fasta -R 75 -o hg19_allExon_simulated --pe 500 50 --vcf -t SSAllExon_V6_r2_S07604514_covered.bed -c 100 --job 2 23

And the error is:

image

Thanks

zstephens commented 5 years ago

Greetings, This error suggests that the system genReads.py is running on is unable to allocate the RAM required to store the reference sequence in memory. What kind of system are you running on? Are you trying to run all 23 jobs at once on the same computer (such that your memory is tasked with holding 23 copies of the reference sequences at a time?). -Zach

mrpk123 commented 5 years ago

Hi Zach,

Thank you for your prompt reply 😊

I am running 1 job at a time. RAM is 16GB. No other applications are running.

Regards Mrinal

From: zstephens notifications@github.com Sent: Thursday, May 23, 2019 1:42 AM To: zstephens/neat-genreads neat-genreads@noreply.github.com Cc: Mrinal Puranik mrinal_puranik@persistent.com; Author author@noreply.github.com Subject: Re: [zstephens/neat-genreads] Getting memory error while using hg19 as reference and targeted bed file for whole exome (#61)

Greetings, This error suggests that the system genReads.py is running on is unable to allocate the RAM required to store the reference sequence in memory. What kind of system are you running on? Are you trying to run all 23 jobs at once on the same computer (such that your memory is tasked with holding 23 copies of the reference sequences at a time?). -Zach

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zstephens/neat-genreads/issues/61?email_source=notifications&email_token=AMEMJBJWHNYXM754AHJGWSDPWWSIFA5CNFSM4HOR7RX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWAGCVA#issuecomment-494952788, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMEMJBPP76MFRXUBNDDYKOLPWWSIFANCNFSM4HOR7RXQ. DISCLAIMER

This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

zstephens commented 5 years ago

Hmm. I tried to replicate this but was unable to. (granted I used a different BED file to specify exome region: https://support.illumina.com/downloads/truseq-exome-product-files.html )

Are you able to track memory usage through an activity monitor, or something comparable, to see what its usage gets up to before dying?

Is there anything particularly funky about the BED file you're using? If that file is publicly available (or if you don't mind sharing it with me) I could attempt to run the exact command on my end to see if there's a bug of some sort.

mrpk123 commented 5 years ago

Hi,

I tried the bed file from the link provided by you and it is working fine albeit taking long time for each job. The bed file that I am using can be downloaded from Agilent’s SureDesign site. The design is SureSelectHuman All Exon V6 r2 and reference genome is hg19

Thanks

From: zstephens notifications@github.com Sent: Friday, May 24, 2019 9:08 PM To: zstephens/neat-genreads neat-genreads@noreply.github.com Cc: Mrinal Puranik mrinal_puranik@persistent.com; Author author@noreply.github.com Subject: Re: [zstephens/neat-genreads] Getting memory error while using hg19 as reference and targeted bed file for whole exome (#61)

Hmm. I tried to replicate this but was unable to. (granted I used a different BED file to specify exome region: https://support.illumina.com/downloads/truseq-exome-product-files.html )

Are you able to track memory usage through an activity monitor, or something comparable, to see what its usage gets up to before dying?

Is there anything particularly funky about the BED file you're using? If that file is publicly available (or if you don't mind sharing it with me) I could attempt to run the exact command on my end to see if there's a bug of some sort.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zstephens/neat-genreads/issues/61?email_source=notifications&email_token=AMEMJBOY46SEQHS7TIZAEL3PXADU3A5CNFSM4HOR7RX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWFX3PQ#issuecomment-495680958, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMEMJBMIDVDV5SVZGLEZPTTPXADU3ANCNFSM4HOR7RXQ. DISCLAIMER

This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.