search

jts / nanopolish

Signal-level algorithms for MinION data
MIT License
567 stars 159 forks source link

empty variants (nanopolish.results) file #103

Closed zascavage closed 7 years ago

zascavage commented 7 years ago

I use: python /home/nanopore/nanopolish/scripts/nanopolish_makerange.py rCRS.fasta | parallel --results nanopolish.results -P 8 \ nanopolish variants --consensus polished.{1}.fa -w {1} -r sample.fasta -b sample.sorted.bam -g reference.fasta -e sample.eventalign.sorted.bam -t 4 --min-candidate-frequency 0.1 --models nanopolish_models.fofn to call variants and generate a consensus sequence. I am able to get a consensus every time, however, with results from the 1D rapid kits, the file with the list of variants is empty. I don't have this issue with results from the 2D kits. I have updated my version of nanopolish, and it doesn't seem to have helped. I tried changing --consensus to --snps, I tried running only nanopolish variants. I am no good at troubleshooting, as you know, so can you help me with a command that would give me the variant list? Thanks!

jts commented 7 years ago

Can you try adding a -o (output file) option to the command? I suggest:

python /home/nanopore/nanopolish/scripts/nanopolish_makerange.py rCRS.fasta | parallel --results nanopolish.results -P 8 \ nanopolish variants --consensus polished.{1}.fa -o polished.{1}.vcf -w {1} -r sample.fasta -b sample.sorted.bam -g reference.fasta -e sample.eventalign.sorted.bam -t 4 --min-candidate-frequency 0.1 --models nanopolish_models.fofn
zascavage commented 7 years ago

Unfortunately, that did not do it... it did change the output files though. Traditionally, the polished.gi file that is created has the consensus sequence in it. This time, it had the header of the nanopolish.results stdout, but just as the stdout file, there is nothing but a header.

It just seems odd that it is specific to the type of sequencing reaction. Both have had local basecalling, as well as metrichor basecalling, and that doesn't seem to make a difference. The 1d rapid result files will output all the correct files with data in them, and the 2d barcoded result files will not give me the list of variants, however it still complies a consensus sequence.

From: Jared Simpson notifications@github.com Sent: Tuesday, January 10, 2017 12:07:06 PM To: jts/nanopolish Cc: Roxanne Zascavage; Author Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Can you try adding a -o (output file) option to the command? I suggest:

python /home/nanopore/nanopolish/scripts/nanopolish_makerange.py rCRS.fasta | parallel --results nanopolish.results -P 8 \ nanopolish variants --consensus polished.{1}.fa -o polished.{1}.vcf -w {1} -r sample.fasta -b sample.sorted.bam -g reference.fasta -e sample.eventalign.sorted.bam -t 4 --min-candidate-frequency 0.1 --models nanopolish_models.fofn

- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/nanopolish/issues/103#issuecomment-271651150, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH6v7JE-X1p2Vl2onOwH5iMmib7PrZ9Iks5rQ8jKgaJpZM4Lfp9Z.

zascavage commented 7 years ago

Hey Jared, Sorry to keep bugging you about this, but is there something I am missing for compatibility with 1D reads? The more I think about it and try alternatives, I feel like this is what my issue boils down to. When I look in the help, it says --reads is for the 2D file where the ONT reads are located. I am also still working off a protocol that used bwa mem –x ont2d… etc. earlier in the pipeline for data extraction. Maybe this is where my problem is? I am not sure, so any other suggestions would be very helpful! I can also send you a data file and see if you can get it to work??

THANKS!

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu| www.unthsc.eduhttp://www.unthsc.edu/

From: Jared Simpson [mailto:notifications@github.com] Sent: Tuesday, January 10, 2017 12:07 PM To: jts/nanopolish nanopolish@noreply.github.com Cc: Roxanne Zascavage rmceowen@live.unthsc.edu; Author author@noreply.github.com Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Can you try adding a -o (output file) option to the command? I suggest:

python /home/nanopore/nanopolish/scripts/nanopolish_makerange.py rCRS.fasta | parallel --results nanopolish.results -P 8 \ nanopolish variants --consensus polished.{1}.fa -o polished.{1}.vcf -w {1} -r sample.fasta -b sample.sorted.bam -g reference.fasta -e sample.eventalign.sorted.bam -t 4 --min-candidate-frequency 0.1 --models nanopolish_models.fofn

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/nanopolish/issues/103#issuecomment-271651150, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH6v7JE-X1p2Vl2onOwH5iMmib7PrZ9Iks5rQ8jKgaJpZM4Lfp9Z.

jts commented 7 years ago

Are you using poretools to extract the reads from the fast5 files? Sometimes if there is a combination of metrichor and local basecalling applied to the fast5, it can cause problems. Could you try running nanopolish extract to export the sequences, then try your pipeline again?

zascavage commented 7 years ago

I was using poretools. I tried nanopolish extract, but it only gives me an empty file as well (at least for my 1D runs). It appears that is the issue: it is not compatible with 1D reads… Boo. Thanks for your help! I’ll keep an eye out for a 1D compatible update.

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu| www.unthsc.eduhttp://www.unthsc.edu/

From: Jared Simpson [mailto:notifications@github.com] Sent: Wednesday, January 25, 2017 7:01 PM To: jts/nanopolish nanopolish@noreply.github.com Cc: Roxanne Zascavage rmceowen@live.unthsc.edu; Author author@noreply.github.com Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Are you using poretools to extract the reads from the fast5 files? Sometimes if there is a combination of metrichor and local basecalling applied to the fast5, it can cause problems. Could you try running nanopolish extract to export the sequences, then try your pipeline again?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/nanopolish/issues/103#issuecomment-275281016, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH6v7Pv1Gkyfs7lUSzWtHZF3axcu6TH8ks5rV_BEgaJpZM4Lfp9Z.

jts commented 7 years ago

nanopolish extract should be compatible with 1D reads - what basecaller did you use?

Jared

On Mon, Jan 30, 2017 at 2:33 PM, zascavage notifications@github.com wrote:

I was using poretools. I tried nanopolish extract, but it only gives me an empty file as well (at least for my 1D runs). It appears that is the issue: it is not compatible with 1D reads… Boo. Thanks for your help! I’ll keep an eye out for a 1D compatible update.

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 <(817)%20735-2953> roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu| www.unthsc.eduhttp://www.unthsc.edu/

From: Jared Simpson [mailto:notifications@github.com] Sent: Wednesday, January 25, 2017 7:01 PM To: jts/nanopolish nanopolish@noreply.github.com Cc: Roxanne Zascavage rmceowen@live.unthsc.edu; Author < author@noreply.github.com> Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Are you using poretools to extract the reads from the fast5 files? Sometimes if there is a combination of metrichor and local basecalling applied to the fast5, it can cause problems. Could you try running nanopolish extract to export the sequences, then try your pipeline again?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/ nanopolish/issues/103#issuecomment-275281016, or mute the thread< https://github.com/notifications/unsubscribe-auth/ AH6v7Pv1Gkyfs7lUSzWtHZF3axcu6TH8ks5rV_BEgaJpZM4Lfp9Z>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jts/nanopolish/issues/103#issuecomment-276165556, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXxn_lJt5YhRjXnmApzWQpbP6AfkPwrks5rXjr9gaJpZM4Lfp9Z .

zascavage commented 7 years ago

The Nanopore local basecaller…

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu| www.unthsc.eduhttp://www.unthsc.edu/

From: Jared Simpson [mailto:notifications@github.com] Sent: Monday, January 30, 2017 1:52 PM To: jts/nanopolish nanopolish@noreply.github.com Cc: Roxanne Zascavage rmceowen@live.unthsc.edu; Author author@noreply.github.com Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

nanopolish extract should be compatible with 1D reads - what basecaller did you use?

Jared

On Mon, Jan 30, 2017 at 2:33 PM, zascavage notifications@github.com<mailto:notifications@github.com> wrote:

I was using poretools. I tried nanopolish extract, but it only gives me an empty file as well (at least for my 1D runs). It appears that is the issue: it is not compatible with 1D reads… Boo. Thanks for your help! I’ll keep an eye out for a 1D compatible update.

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 <(817)%20735-2953> roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu|mailto:roxanne.zascavage@unthsc.edu%3cmailto:roxanne.zascavage@unthsc.edu%3e| www.unthsc.eduhttp://www.unthsc.edu/<http://www.unthsc.edu%3chttp:/www.unthsc.edu/>

From: Jared Simpson [mailto:notifications@github.com] Sent: Wednesday, January 25, 2017 7:01 PM To: jts/nanopolish nanopolish@noreply.github.com<mailto:nanopolish@noreply.github.com> Cc: Roxanne Zascavage rmceowen@live.unthsc.edu<mailto:rmceowen@live.unthsc.edu>; Author < author@noreply.github.commailto:author@noreply.github.com> Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Are you using poretools to extract the reads from the fast5 files? Sometimes if there is a combination of metrichor and local basecalling applied to the fast5, it can cause problems. Could you try running nanopolish extract to export the sequences, then try your pipeline again?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/ <https://github.com/jts/%0b> nanopolish/issues/103#issuecomment-275281016>, or mute the thread< https://github.com/notifications/unsubscribe-auth/ AH6v7Pv1Gkyfs7lUSzWtHZF3axcu6TH8ks5rV_BEgaJpZM4Lfp9Z>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jts/nanopolish/issues/103#issuecomment-276165556, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXxn_lJt5YhRjXnmApzWQpbP6AfkPwrks5rXjr9gaJpZM4Lfp9Z .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/nanopolish/issues/103#issuecomment-276170907, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH6v7JhNvdd0EQ8J2sZ-bUqA894nmZnvks5rXj9ugaJpZM4Lfp9Z.

jts commented 7 years ago

Ok, I think that might be the problem. Another user reported problems with albacore basecalls, I'll look into it soon.

Jared

On Tue, Jan 31, 2017 at 9:51 AM, zascavage notifications@github.com wrote:

The Nanopore local basecaller…

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 <(817)%20735-2953> roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu| www.unthsc.eduhttp://www.unthsc.edu/

From: Jared Simpson [mailto:notifications@github.com] Sent: Monday, January 30, 2017 1:52 PM To: jts/nanopolish nanopolish@noreply.github.com Cc: Roxanne Zascavage rmceowen@live.unthsc.edu; Author < author@noreply.github.com> Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

nanopolish extract should be compatible with 1D reads - what basecaller did you use?

Jared

On Mon, Jan 30, 2017 at 2:33 PM, zascavage <notifications@github.com< mailto:notifications@github.com>> wrote:

I was using poretools. I tried nanopolish extract, but it only gives me an empty file as well (at least for my 1D runs). It appears that is the issue: it is not compatible with 1D reads… Boo. Thanks for your help! I’ll keep an eye out for a 1D compatible update.

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 <(817)%20735-2953> <(817)%20735-2953> roxanne.zascavage@unthsc.edu<mailto:roxanne.zascavage@unthsc.edu |<mailto:roxanne.zascavage@unthsc.edu%3cmailto: roxanne.zascavage@unthsc.edu%3e|> www.unthsc.eduhttp://www.unthsc.edu/<http://www.unthsc.edu%3chttp:/ www.unthsc.edu/>

From: Jared Simpson [mailto:notifications@github.com] Sent: Wednesday, January 25, 2017 7:01 PM To: jts/nanopolish <nanopolish@noreply.github.com<mailto: nanopolish@noreply.github.com>> Cc: Roxanne Zascavage <rmceowen@live.unthsc.edu<mailto: rmceowen@live.unthsc.edu>>; Author < author@noreply.github.commailto:author@noreply.github.com> Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Are you using poretools to extract the reads from the fast5 files? Sometimes if there is a combination of metrichor and local basecalling applied to the fast5, it can cause problems. Could you try running nanopolish extract to export the sequences, then try your pipeline again?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/ <https://github.com/jts/%0b> nanopolish/issues/103#issuecomment-275281016>, or mute the thread< https://github.com/notifications/unsubscribe-auth/ AH6v7Pv1Gkyfs7lUSzWtHZF3axcu6TH8ks5rV_BEgaJpZM4Lfp9Z>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jts/nanopolish/issues/103#issuecomment-276165556, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXxn_ lJt5YhRjXnmApzWQpbP6AfkPwrks5rXjr9gaJpZM4Lfp9Z .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/ nanopolish/issues/103#issuecomment-276170907, or mute the thread< https://github.com/notifications/unsubscribe-auth/AH6v7JhNvdd0EQ8J2sZ- bUqA894nmZnvks5rXj9ugaJpZM4Lfp9Z>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jts/nanopolish/issues/103#issuecomment-276383787, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXxn7wMjo2bW7W4eEnYS5eLm9iMufirks5rX0p8gaJpZM4Lfp9Z .

zascavage commented 7 years ago

Thanks! You’re amazing, by the way

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu| www.unthsc.eduhttp://www.unthsc.edu/

From: Jared Simpson [mailto:notifications@github.com] Sent: Tuesday, January 31, 2017 9:13 AM To: jts/nanopolish nanopolish@noreply.github.com Cc: Roxanne Zascavage rmceowen@live.unthsc.edu; Author author@noreply.github.com Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Ok, I think that might be the problem. Another user reported problems with albacore basecalls, I'll look into it soon.

Jared

On Tue, Jan 31, 2017 at 9:51 AM, zascavage notifications@github.com<mailto:notifications@github.com> wrote:

The Nanopore local basecaller…

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 <(817)%20735-2953> roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu|mailto:roxanne.zascavage@unthsc.edu%3cmailto:roxanne.zascavage@unthsc.edu%3e| www.unthsc.eduhttp://www.unthsc.edu/<http://www.unthsc.edu%3chttp:/www.unthsc.edu/>

From: Jared Simpson [mailto:notifications@github.com] Sent: Monday, January 30, 2017 1:52 PM To: jts/nanopolish nanopolish@noreply.github.com<mailto:nanopolish@noreply.github.com> Cc: Roxanne Zascavage rmceowen@live.unthsc.edu<mailto:rmceowen@live.unthsc.edu>; Author < author@noreply.github.commailto:author@noreply.github.com> Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

nanopolish extract should be compatible with 1D reads - what basecaller did you use?

Jared

On Mon, Jan 30, 2017 at 2:33 PM, zascavage <notifications@github.com< mailto:notifications@github.com%3c%0b> mailto:notifications@github.com>> wrote:

I was using poretools. I tried nanopolish extract, but it only gives me an empty file as well (at least for my 1D runs). It appears that is the issue: it is not compatible with 1D reads… Boo. Thanks for your help! I’ll keep an eye out for a 1D compatible update.

Roxanne R Zascavage Post Doctoral Associate | Molecular and Medical Genetics University of North Texas health Science Center | Office: CBH 356 3500 Camp Bowie Blvd | Fort Worth, TX 76107 817-735-2953 <(817)%20735-2953> <(817)%20735-2953> roxanne.zascavage@unthsc.edumailto:roxanne.zascavage@unthsc.edu<mailto:roxanne.zascavage@unthsc.edu%3cmailto:roxanne.zascavage@unthsc.edu |<mailto:roxanne.zascavage@unthsc.edu%3cmailto: mailto:roxanne.zascavage@unthsc.edu%3cmailto:%0b> roxanne.zascavage@unthsc.edu%3e|mailto:roxanne.zascavage@unthsc.edu%3e|> www.unthsc.eduhttp://www.unthsc.edu/<http://www.unthsc.edu%3chttp:/<http://www.unthsc.edu%3chttp:/www.unthsc.edu/%3chttp:/www.unthsc.edu%3chttp:/ www.unthsc.edu/http://www.unthsc.edu/>>

From: Jared Simpson [mailto:notifications@github.com] Sent: Wednesday, January 25, 2017 7:01 PM To: jts/nanopolish <nanopolish@noreply.github.com<mailto: mailto:nanopolish@noreply.github.com%3cmailto:%0b> nanopolish@noreply.github.commailto:nanopolish@noreply.github.com>> Cc: Roxanne Zascavage <rmceowen@live.unthsc.edu<mailto: mailto:rmceowen@live.unthsc.edu%3cmailto:%0b> rmceowen@live.unthsc.edumailto:rmceowen@live.unthsc.edu>>; Author < author@noreply.github.commailto:author@noreply.github.com<mailto:author@noreply.github.com%3cmailto:author@noreply.github.com>> Subject: Re: [jts/nanopolish] empty variants (nanopolish.results) file (#103)

Are you using poretools to extract the reads from the fast5 files? Sometimes if there is a combination of metrichor and local basecalling applied to the fast5, it can cause problems. Could you try running nanopolish extract to export the sequences, then try your pipeline again?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/ <https://github.com/jts/%0b> https://github.com/jts/%0b> nanopolish/issues/103#issuecomment-275281016>, or mute the thread< https://github.com/notifications/unsubscribe-auth/ AH6v7Pv1Gkyfs7lUSzWtHZF3axcu6TH8ks5rV_BEgaJpZM4Lfp9Z>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jts/nanopolish/issues/103#issuecomment-276165556, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXxn_ <https://github.com/notifications/unsubscribe-auth/AAXxn_%0b> lJt5YhRjXnmApzWQpbP6AfkPwrks5rXjr9gaJpZM4Lfp9Z> .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/ <https://github.com/jts/%0b> nanopolish/issues/103#issuecomment-276170907>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AH6v7JhNvdd0EQ8J2sZ- bUqA894nmZnvks5rXj9ugaJpZM4Lfp9Z>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jts/nanopolish/issues/103#issuecomment-276383787, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXxn7wMjo2bW7W4eEnYS5eLm9iMufirks5rX0p8gaJpZM4Lfp9Z .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jts/nanopolish/issues/103#issuecomment-276389886, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH6v7I1AWT9DDMF5YKDsB6PlNdJuJjnJks5rX09_gaJpZM4Lfp9Z.

jts commented 7 years ago

Hi Roxanne,

The latest version should work with albacore basecalls.

Jared