During clustering, `N` should be treated as a mismatch.

akikuno commented 3 hours ago

📋 Description

Currently, in clustering.score_handler.annotate_score, samples with a large number of N are being incorrectly judged as having no mismatches. Since N is clearly not a match, it must be accounted for in the scoring.

{"QNAME": "0035f39c-b841-4df3-8390-21c4868736cc", "RNAME": "control", "CSSPLIT": "=A,=A,=A,=T,+G|=G,=G,=G,=C,=G,=G,=T,=A,=G,=G,=C,=G,=T,=G,=T,=A,=C,=G,=G,=T,-G,-G,=G,=A,=G,=G,=T,=C,=T,=A,=T,=A,=T,=A,=A,=G,=C,=A,=G,=A,=G,=G,=T,=C,=G,=T,=T,=T,=A,=G,=T,=G,=A,=A,=C,=C,=G,=T,=C,=A,=G,=A,=T,=C,=A,=C,=T,=A,=G,=T,*AG,=G,=C,=T,=T,-T,-A,*TC,=T,=G,=C,=G,=G,=T,=A,=G,+T|=T,=T,=T,*AC,=T,=C,=A,=C,=A,=G,=T,=T,=A,=A,=A,=T,=T,=G,=C,=T,=A,=A,=C,*GA,=C,+A|=A,=G,+A|+A|+A|+A|=T,*CA,=A,+G|=G,=T,=G,=C,=T,=C,=G,=A,=C,=T,=G,=A,=T,=C,=A,=C,=A,=G,=G,=T,=A,=A,=G,=T,=A,=T,=C,=A,=A,=G,=G,=T,=T,=A,=C,=A,=A,=G,=A,=C,=A,=G,*GT,=T,+C|=T,=T,*AC,=A,-G,=G,=A,=G,+G|+C|=G,=C,=C,=A,=A,=T,=A,=G,+A|=A,=A,=A,=C,=T,=G,=G,=G,=C,=T,=T,=G,=T,=C,=G,=A,=G,=A,=C,=A,=G,=A,=G,=A,=A,=G,=A,=T,=T,=C,=T,=T,=G,=C,=G,=T,=T,*TA,=C,=T,=G,=A,=T,=A,=G,=G,=C,=A,=C,=C,=T,=A,=T,=T,=G,=G,=T,=C,=T,=T,=A,=C,=T,=G,=A,=C,=A,=T,=C,=C,=A,=C,-T,=T,=T,=G,=C,=C,=T,=T,-T,-C,=T,=C,=T,=C,=C,=A,=C,=A,=G,=G,-G,-G,-T,=A,*CT,=C,+T|+T|=G,=A,=A,=G,+A|+A|=C,=C,=G,=C,=T,=A,=G,=C,=G,=C,=T,=A,=C,=C,-G,=G,=T,*CA,=G,=C,=C,=A,=C,=C,=A,=T,=G,-C,=C,=C,=G,=C,=C,=A,=T,=G,=A,+A|+G|+G|=A,=G,=A,=T,=C,=G,=A,=G,=T,=G,=C,=C,=G,=C,=A,=T,=C,=A,=C,=C,=G,=G,=C,=A,=C,=C,=C,=T,=G,=A,=A,=C,=G,=G,=C,=G,=T,=G,=G,=A,=G,=T,=T,-C,=G,=A,=G,=C,=T,=G,=G,=T,=G,=G,=G,=C,=G,=G,=C,=G,=G,=A,=G,=A,=G,=G,=G,=C,=A,-C,=C,=C,=C,*CA,=G,=A,=G,=C,=A,=G,=G,=G,=C,=C,=G,=C,=A,=T,=G,=A,=C,=C,=A,=A,=C,-A,=A,=G,=A,=T,+G|=G,=A,=A,=G,+A|=A,=G,=C,=A,+C|=C,=C,=A,=A,=A,=G,=G,=C,=G,-C,=C,=C,=T,=G,=A,=C,=C,=T,=T,-C,=A,=G,=C,=C,=C,=C,=T,=A,=C,=C,=T,=G,=C,=T,=G,=A,=G,=C,=C,=A,+G|=C,=G,=T,=G,=A,=T,=G,=G,=G,=C,=T,=A,=C,=G,=G,=C,=T,=T,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N", "STRAND": "+"}

💬 Current Behavior

No response

🎯 Expected Behavior

No response

⚠️ Error message

No response

🔍 Environment

- OS:
- DAJIN2 version: 0.5.5
- Python version:

📎 Anything else?

No response

akikuno commented 2 hours ago

First, it might be a good idea to perform anomaly detection on MIDSV during the preprocessing stage for control, and exclude abnormal sequences (including samples with many Ns).

As for the sample, it may be effective to use the abnormal sequences found in control as a reference and exclude similar sequences from the sample.

akikuno commented 1 hour ago

Output reads with many sequence errors in the control as BAM/control/sequence_errors.bam, and exclude them from the analysis.

Additionally, exclude reads from samples that are similar to those in BAM/control/sequence_errors.bam from the analysis as well, outputting them as BAM/sample/sequence_errors.bam.

akikuno / DAJIN2