search

amplab / snap

Scalable Nucleotide Alignment Program -- a fast and accurate read aligner for high-throughput sequencing data
https://www.microsoft.com/en-us/research/project/snap/
Apache License 2.0
287 stars 66 forks source link

Bad error message for too-long reads #138

Closed YasirKusay closed 2 years ago

YasirKusay commented 2 years ago

Hi, I am using snap-aligner to run this command: snap-aligner single index input.fq -o output.sam

and have got this error:

Loading index from directory... 0s.  86283964 bases, seed size 20
Aligning.
Failed to write into fresh buffer; trying providing the -wbs switch with a larger value
SNAP exited with exit code 1 from line 236 of file SNAPLib/ReadWriter.cpp

Some context: index is a small index created from viral.1.1.genomic.fna and input.fq is an assembly file (also relatively small). In addition, the assembly file was originally outputted in fasta format but I have converted it to fastq using dummy quality values (composed of '#' characters).

Can you help me out please?

bolosky commented 2 years ago

That happens when the set of reads written by a single input read pair is too big to fit into an output buffer (which is 16MB by default). The normal reason you'd see this is if a read pair produced lots of alternate alignments, but I suppose it could also happen if you made really big reads or something.

You could try rerunning snap giving it the -wbs switch with some value larger than the default 16 and see if that helps.

What version are you running? It doesn't look like the current one based on the line number where it exited. I think you might have a pretty old one, so you might want to update to the newest 1.0.4 version.

Also, if it still doesn't work, send me the command line you used to run snap.

--Bill

From: YasirKusay @.> Sent: Saturday, November 6, 2021 4:46 AM To: amplab/snap @.> Cc: Subscribed @.***> Subject: [amplab/snap] -wbs Issue (Issue #138)

Hi, I am using snap-aligner to run this command: snap-aligner single index input.fq -o output.sam

and have got this error:

Loading index from directory... 0s. 86283964 bases, seed size 20

Aligning.

Failed to write into fresh buffer; trying providing the -wbs switch with a larger value

SNAP exited with exit code 1 from line 236 of file SNAPLib/ReadWriter.cpp

Some context: index is a small index created from viral.1.1.genomic.fna and input.fq is an assembly file (also relatively small). In addition, the assembly file was originally outputted in fasta format but I have converted it to fastq using dummy quality values (composed of '#' characters).

Can you help me out please?

- You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Famplab%2Fsnap%2Fissues%2F138&data=04%7C01%7Cbolosky%40microsoft.com%7C5f1b1726bf4b444b65a408d9a11b0dca%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637717959849080885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=x6Z1gl0H6po%2B7eUl4%2FUjUtKib9D%2FZFhaou6MZ12M1fQ%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAHPTWKLLQ7ZCCXY6HPXYJLUKUIQ3ANCNFSM5HPQZBOQ&data=04%7C01%7Cbolosky%40microsoft.com%7C5f1b1726bf4b444b65a408d9a11b0dca%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637717959849090833%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FJg9T0%2FXTYMfZX98945Dp%2BI5WBa3h4xmRJ%2F9De0izFg%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cbolosky%40microsoft.com%7C5f1b1726bf4b444b65a408d9a11b0dca%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637717959849090833%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jXry6U2GVIxKkWoVhDV9U09wQ3KlszDauqi1R2k%2B9rQ%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cbolosky%40microsoft.com%7C5f1b1726bf4b444b65a408d9a11b0dca%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637717959849100791%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tGfhlskFebtDmpPgFr8UG7Pnr2QzQalUEFO0iTglRl8%3D&reserved=0.

YasirKusay commented 2 years ago

Hi @bolosky, thank you for your reply. The read size are no greater than 700 bp for the sample I am using and I am using version 1.0.23 beta. While I did try to increase the size of -wbs (to 184), to no success, I will try again but with larger computational resources.

bolosky commented 2 years ago

No, something else is going on. You shouldn't need more computational resources for that.

I'd try running the current version of SNAP (1.0.4). If that doesn't work, then if you can share your reference and read sets for me I'll try to repro it and see what the problem.

From: YasirKusay @.> Sent: Saturday, November 6, 2021 11:54 PM To: amplab/snap @.> Cc: Bill Bolosky @.>; Mention @.> Subject: Re: [amplab/snap] -wbs Issue (Issue #138)

Hi @boloskyhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbolosky&data=04%7C01%7Cbolosky%40microsoft.com%7C8db3954a12e44d2c88da08d9a1bb54f4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637718648241088721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=89tnUsXodYDy9mMEgaGJLgcO9MA7Olo7wq1CKgtcxPA%3D&reserved=0, thank you for your reply. The read size are no greater than 700 bp for the sample I am using and I am using version 1.0.23 beta. While I did try to increase the size of -wbs (to 184), to no success, I will try again but with larger computational resources.

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Famplab%2Fsnap%2Fissues%2F138%23issuecomment-962561148&data=04%7C01%7Cbolosky%40microsoft.com%7C8db3954a12e44d2c88da08d9a1bb54f4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637718648241098678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xU8Tu02d3eEnYTiCXQN%2Fa972bflPyQSbC5JVqf9dlTk%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAHPTWOCQNROMQSANAXAIM3UKYO7HANCNFSM5HPQZBOQ&data=04%7C01%7Cbolosky%40microsoft.com%7C8db3954a12e44d2c88da08d9a1bb54f4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637718648241098678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eqhVpw%2BmGghjl9jceFOUaoTvRfOFZn8bq960aJQAT8o%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cbolosky%40microsoft.com%7C8db3954a12e44d2c88da08d9a1bb54f4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637718648241108633%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=s9hbto8FfCWMg29xKhPv6HbOZoRwaFbza4rHzHKA3FY%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cbolosky%40microsoft.com%7C8db3954a12e44d2c88da08d9a1bb54f4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637718648241108633%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oFDXVLjqEOdKrouRm8z0esXNM1u0WNN0VgzYI8Vdscs%3D&reserved=0.

YasirKusay commented 2 years ago

I will try that, thank you

YasirKusay commented 2 years ago

Hi @bolosky , I installed SNAP version 1.0.4 (although I am greeted with "Welcome to SNAP version 1.0.3" when just running snap-aligner). Unfortunately, I am still getting the same issue despite increasing the -wbs size. Can you help me out? Also there are 3 reads that are above 1000 bp in length, (the largest of them being almost 4000 bp long).

bolosky commented 2 years ago

What's happening here is that the input has at least one read longer than the compiled-in maximum read length and it's not properly producing an error message. We need to fix the error handling.

YasirKusay commented 2 years ago

Hi Bill,

Thank you for your reply, I will now attempt to run it again with a larger read value.

Kind Regards,

Yasir

From: Bill Bolosky @.> Sent: Tuesday, 9 November 2021 12:35 PM To: amplab/snap @.> Cc: YasirKusay @.>; Author @.> Subject: Re: [amplab/snap] -wbs Issue (Issue #138)

What's happening here is that the input has at least one read longer than the compiled-in maximum read length and it's not properly producing an error message. We need to fix the error handling.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/amplab/snap/issues/138#issuecomment-963732539, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOIKS7AVG4QMRFOLMP7W2UTULB3FDANCNFSM5HPQZBOQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

bolosky commented 2 years ago

Fixed in 2.0