Closed jorvis closed 6 years ago
Hi,
Could you please try larger s parameter? Thanks!
On Sun, Nov 12, 2017 at 11:29 PM, Joshua Orvis notifications@github.com wrote:
I was able to format smaller FASTA files for searching using prerapsearch on my system, but when I try to do the latest Uniref100 it fails at various times. Some attempts have run as long as 6 hours, generating a 204GB index before failing. Retries have it fail within just a few minutes. I have not run out of disk space and it happens whether I use the -s option or not.
The input FASTA file in this case is 56GB. I get very little output to work with.
$ ./RAPSearch2.24_64bits/bin/prerapsearch -d uniref100.20171025.fasta -n uniref100.20171025.rapsearch2.db -s 10 now building hash file Killed
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38, or mute the thread https://github.com/notifications/unsubscribe-auth/AETkp4cwhLTja4TI55ZihFFRdc9FYJv_ks5s18WdgaJpZM4QbOwB .
-- Sincerely, Y. Z.
I tried up to -s 40 with the same result. What's a practical upper limit?
On Nov 15, 2017 8:07 AM, "zhaoyanswill" notifications@github.com wrote:
Hi,
Could you please try larger s parameter? Thanks!
On Sun, Nov 12, 2017 at 11:29 PM, Joshua Orvis notifications@github.com wrote:
I was able to format smaller FASTA files for searching using prerapsearch on my system, but when I try to do the latest Uniref100 it fails at various times. Some attempts have run as long as 6 hours, generating a 204GB index before failing. Retries have it fail within just a few minutes. I have not run out of disk space and it happens whether I use the -s option or not.
The input FASTA file in this case is 56GB. I get very little output to work with.
$ ./RAPSearch2.24_64bits/bin/prerapsearch -d uniref100.20171025.fasta -n uniref100.20171025.rapsearch2.db -s 10 now building hash file Killed
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38, or mute the thread https://github.com/notifications/unsubscribe-auth/ AETkp4cwhLTja4TI55ZihFFRdc9FYJv_ks5s18WdgaJpZM4QbOwB .
-- Sincerely, Y. Z.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38#issuecomment-344602049, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUMk_9xI1j8QFvNXTYih_XzfoTE2ukkks5s2u_qgaJpZM4QbOwB .
Could you please try an even larger number, for example, 200, and then see if it's killed? Thanks!
On Wed, Nov 15, 2017 at 9:08 AM, Joshua Orvis notifications@github.com wrote:
I tried up to -s 40 with the same result. What's a practical upper limit?
On Nov 15, 2017 8:07 AM, "zhaoyanswill" notifications@github.com wrote:
Hi,
Could you please try larger s parameter? Thanks!
On Sun, Nov 12, 2017 at 11:29 PM, Joshua Orvis <notifications@github.com
wrote:
I was able to format smaller FASTA files for searching using prerapsearch on my system, but when I try to do the latest Uniref100 it fails at various times. Some attempts have run as long as 6 hours, generating a 204GB index before failing. Retries have it fail within just a few minutes. I have not run out of disk space and it happens whether I use the -s option or not.
The input FASTA file in this case is 56GB. I get very little output to work with.
$ ./RAPSearch2.24_64bits/bin/prerapsearch -d uniref100.20171025.fasta -n uniref100.20171025.rapsearch2.db -s 10 now building hash file Killed
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38, or mute the thread https://github.com/notifications/unsubscribe-auth/ AETkp4cwhLTja4TI55ZihFFRdc9FYJv_ks5s18WdgaJpZM4QbOwB .
-- Sincerely, Y. Z.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38#issuecomment- 344602049, or mute the thread https://github.com/notifications/unsubscribe- auth/AAUMk_9xI1j8QFvNXTYih_XzfoTE2ukkks5s2u_qgaJpZM4QbOwB .
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38#issuecomment-344602642, or mute the thread https://github.com/notifications/unsubscribe-auth/AETkp85h3BafumbeTUG08gCxd8CRCqeTks5s2vBmgaJpZM4QbOwB .
-- Sincerely, Y. Z.
It worked with "-s 100" (wanted to try stepping up.) What are the drawbacks of this?
No drawback if each piece of data ( 204GB/100 in your case) is not very small. '-s' parameter denotes how many pieces are data is divided to fit in the memory of the machine.
On Wed, Nov 15, 2017 at 2:45 PM, Joshua Orvis notifications@github.com wrote:
It worked with "-s 100" (wanted to try stepping up.) What are the drawbacks of this?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38#issuecomment-344706769, or mute the thread https://github.com/notifications/unsubscribe-auth/AETkp8ExPsP1OYIMKpFizl0Vu3udWTehks5s2z9tgaJpZM4QbOwB .
-- Sincerely, Y. Z.
Thanks for the info. Is that how much memory is needed at index time or search time? I want to keep the search memory low still, so should I only build on a machine similar to my target execution machines with regard to RAM?
Yes, you are right. Thanks!
On Wed, Nov 15, 2017 at 3:44 PM, Joshua Orvis notifications@github.com wrote:
Thanks for the info. Is that how much memory is needed at index time or search time? I want to keep the search memory low still, so should I only build on a machine similar to my target execution machines with regard to RAM?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/zhaoyanswill/RAPSearch2/issues/38#issuecomment-344722502, or mute the thread https://github.com/notifications/unsubscribe-auth/AETkp9VxbuuPZl369-GZbAVeVEJnCmRXks5s200dgaJpZM4QbOwB .
-- Sincerely, Y. Z.
I was able to format smaller FASTA files for searching using prerapsearch (v 2.24) on my system (Ubuntu 17.04), but when I try to do the latest Uniref100 it fails at various times. Some attempts have run as long as 6 hours, generating a 204GB index before failing. Retries have it fail within just a few minutes. I have not run out of disk space and it happens whether I use the -s option or not.
The input FASTA file in this case is 56GB. I get very little output to work with.