search

haotianteng / Chiron

A basecaller for Oxford Nanopore Technologies' sequencers
Other
122 stars 53 forks source link

Function getcnnfeature() #54

Closed qianlou closed 6 years ago

qianlou commented 6 years ago

Hi Teng, I am afraid we can not discuss at a closed issue section, so I create a new issue with a new name.

I know the getcnnfeature() function will be called whenever rnn layer>0 or not. But from my opinion, getcnnfeature() function does't include any operations such as matrix multiply or convolution operations just some reshape operations. Can getcnnfeature() function get CNN output from signal input? Sincerely, Qian

haotianteng commented 6 years ago

I will close this issue as this is not about the bug either a request feature of Chiron, you are welcome to send me an email or keep discussing under this issue.

As for your question, the following function will construct the demanding CNN graph according to the configuration. net = model_dictcnn_config

On Mon., 28 May 2018, 2:41 pm qianlou, notifications@github.com wrote:

Hi Teng, I am afraid we can not discuss at a closed issue section, so I create a new issue with a new name.

I know the getcnnfeature() function will be called whenever rnn layer>0 or not. But from my opinion, getcnnfeature() function does't include any operations such as matrix multiply or convolution operations just some reshape operations. Can getcnnfeature() function get CNN output from signal input? Sincerely, Qian

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/haotianteng/Chiron/issues/54, or mute the thread https://github.com/notifications/unsubscribe-auth/AKo3X5HJRiDIx5xhOtinERBaJdAmeBOzks5t2_6fgaJpZM4UQNsw .

haotianteng commented 6 years ago

And here is the proof that you can discuss under a closed issue.

qianlou commented 6 years ago

I am sorry that I created a new issue.Yes I should email you and discuss this question. Previously,I am trying to visualize CNN weights of your Chiron model, but I could not find where CNN computations are before. Now I know how did you call conv_layer() function under your help. It's my bad that I am not familiar with python. Thanks for your reply and time, Haotian. Best regards, QianLou

haotianteng commented 6 years ago

Okay, no problem, glad to hear that you solve the problem. It would be interesting to visiualize it, also I have summarize the histogram of the weights, you can inspect it by using the Tensorboard.

On Mon., 28 May 2018, 8:20 pm qianlou, notifications@github.com wrote:

I am sorry that I created a new issue.Yes I should email you and discuss this question. Previously,I am trying to visualize CNN weights of your Chiron model, but I could not find where CNN computations are before. Now I know how did you call conv_layer() function under your help. It's my bad that I am not familiar with python. Thanks for your reply and time, Haotian. Best regards, QianLou

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/haotianteng/Chiron/issues/54#issuecomment-392590080, or mute the thread https://github.com/notifications/unsubscribe-auth/AKo3X5_BRNFeb71T8OcbYAsiL4edmFRMks5t3E4PgaJpZM4UQNsw .

qianlou commented 6 years ago

I just used Tensorboard to profile timing of your model. I don't know it can also profile and visualize weights. I will try it. I am very appreciate your reply. Best regards, Qian

qianlou commented 6 years ago

Hi, Haotian Now I can visualize CNN weights. Thanks for your useful information again.

Also, can you share a Readme file about assess.sh? I am eager to know the accuracy of base callers, which is so important for a new guy in this biology field. I have read your assess.sh and batch_assess.sh scripts. Although I can understand you used graphmap, samtools, and jsa.hts.errorAnalysis and I have installed them, I still can not run this script successfully. However, I can run graphmap and samtools successfully without assess.sh script. when I use jsa.hts.errorAnalysis, there is always a java inner error. Secondly, do you know how I can get a reference file? now I just test command assess ./read1.fasta ./read1.fasta, which means I let read1.fasta as a reference file.

Best regards, QianLou

haotianteng commented 6 years ago

The reference file is in the genomicsresearch server: https://data.genomicsresearch.org/Projects/train_set_all/

Thanks Teng

2018-05-31 10:01 GMT+10:00 qianlou notifications@github.com:

Hi, Haotian Now I can visualize CNN weights. Thanks for your useful information again.

Also, can you share a Readme file about assess.sh? I am eager to know the accuracy of base callers, which is so important for a new guy in this biology field. I have read your assess.sh and batch_assess.sh scripts. Although I can understand you used graphmap, samtools, and jsa.hts.errorAnalysis and I have installed them, I still can not run this script successfully. However, I can run graphmap and samtools successfully without assess.sh script. when I use jsa.hts.errorAnalysis, there is always a java inner error. Secondly, do you know how I can get a reference file? now I just test command assess ./read1.fasta ./read1.fasta, which means I let read1.fasta as a reference file.

Best regards, QianLou

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/haotianteng/Chiron/issues/54#issuecomment-393357617, or mute the thread https://github.com/notifications/unsubscribe-auth/AKo3X8JZWIcH23ENhQ5ZjT_qGdL7u67Yks5t3zLvgaJpZM4UQNsw .

-- Teng Haotian University of Queensland, Queensland, Australia +61 0426116017

haotianteng commented 6 years ago

I just check the assess.sh content, there is a step missing in that file, you have to sort the bam file to make jsa work on it, basically,

samtools sort in.bam in.sorted

and then run jsa.hts.errorAnalysis on the sorted bam file.

qianlou commented 6 years ago

Hi Haotian, I downloaded https://data.genomicsresearch.org/Projects/train_set_all/s10/reference.fasta and https://data.genomicsresearch.org/Projects/train_set_all/s10/pass fast5 files. Then I used trained model to convert fast5 file to fasta file. After that I run the assess.sh assess.txt to get identity using the command: ./assess.sh/IMB14_011406_LT_20170322_FNFAF13375_MN17027_mux_scan_C4_watermang_22032017_75675_ch2_read3_strand.fastq ./Reference/reference.fasta ./** But I only get the NaN result as this: print.txt. Did you see this issue?

qianlou commented 6 years ago

./assess.sh./IMB14_011406_LT_20170322_FNFAF13375_MN17027_mux_scan_C4_watermang_22032017_75675_ch2_read3_strand.fasta ./Reference/reference.fasta ./ has a same result

haotianteng commented 6 years ago

In short, don't use that script, you should be able to use your own custom script to do the assessment, use minimap2, samtools and jsa.

Thanks Teng

2018-06-03 13:45 GMT+10:00 qianlou notifications@github.com:

./assess.sh./IMB14_011406_LT_20170322_FNFAF13375MN17027 mux_scan_C4_watermang_22032017_75675_ch2_read3_strand.fasta ./Reference/reference.fasta ./ has a same result

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/haotianteng/Chiron/issues/54#issuecomment-394132833, or mute the thread https://github.com/notifications/unsubscribe-auth/AKo3XzdcEL8BwlWclPYxC13blqiapxeUks5t41vsgaJpZM4UQNsw .

-- Teng Haotian University of Queensland, Queensland, Australia +61 0426116017