GIAB data

[yk-tanigawa]

• Pacbio data
  • Download
  • select chr 11 and 20 with samtools
• chr11 >20kb
• chr20 >10kb
• start with chr11

[yk-tanigawa]

• total number of reads: 68,064,542
• number of reads with length >= 15kb: 1,821,735
• number of reads with MAPQ >= 50 && length >= 20kb: 344,154

[yk-tanigawa]

I am extracting reads with length >= 15kb It seems like these reads has MAPQ == 50

[yk-tanigawa]

count match/mismatches for the first 100 reads on chr1 in BAM file

id	=	X	I	D	N	S	H	P
1	4182	11019	1635	688	0	63	0	0
2	3883	11532	1225	666	0	311	0	0
3	3921	10627	1061	668	0	71	0	0
4	3806	10965	1092	1178	0	178	0	0
5	4335	10713	911	528	0	3632	0	0
6	3874	11661	2769	781	0	444	0	0
7	4646	13784	2157	1132	0	2899	0	0
8	3519	10667	599	1047	0	518	0	0
9	4705	13738	1408	747	0	327	0	0
10	3721	11239	4655	1122	0	1232	0	0
11	5957	13499	1043	930	0	76	0	0
12	4485	11535	914	684	0	162	0	0
13	4302	10848	768	423	0	5671	0	0
14	4533	11600	1365	404	0	163	0	0
15	20738	294	999	974	0	26	0	0
16	4174	10566	856	984	0	96	0	0
17	4046	10556	980	559	0	264	0	0
18	4546	11210	1057	568	0	131	0	0
19	4174	10461	1144	441	0	140	0	0
20	14535	225	1434	502	0	137	0	0
21	5727	14742	1233	1729	0	343	0	0
22	5353	14809	4394	632	0	967	0	0
23	4289	10096	1157	1081	0	78	0	0
24	5619	13667	1346	993	0	209	0	0
25	3968	9867	912	1596	0	2070	0	0
26	4559	11439	1059	632	0	252	0	0
27	4481	10516	806	905	0	44	0	0
28	4400	13094	1041	1201	0	3014	0	0
29	3562	10198	775	1775	0	1436	0	0
30	4336	10333	957	402	0	88	0	0
31	4148	10197	1115	712	0	207	0	0
32	3973	10663	775	1833	0	3761	0	0
33	4253	11064	1530	517	0	53	0	0
34	3906	11188	950	433	0	216	0	0
35	4084	11152	968	957	0	79	0	0
36	4146	11023	2836	501	0	147	0	0
37	5292	14531	1320	698	0	415	0	0
38	4414	11572	1555	610	0	116	0	0
39	4348	11736	982	1192	0	440	0	0
40	5219	13482	4359	1072	0	180	0	0
41	4836	13238	1153	906	0	380	0	0
42	4126	10925	998	794	0	434	0	0
43	4475	11218	1476	614	0	397	0	0
44	5147	13719	936	1188	0	383	0	0
45	18422	244	1021	919	0	47	0	0
46	15555	317	1535	599	0	57	0	0
47	4257	11300	1695	1306	0	367	0	0
48	6121	15939	2277	974	0	91	0	0
49	5227	12903	1253	493	0	61	0	0
50	5510	14978	889	2380	0	4492	0	0
51	5061	12582	1354	633	0	81	0	0
52	6227	16319	3400	1508	0	252	0	0
53	5535	13688	1122	721	0	86	0	0
54	6496	16755	1376	1010	0	284	0	0
55	4458	11384	3952	1011	0	1122	0	0
56	3976	10115	1817	1014	0	746	0	0
57	4550	11674	1211	401	0	113	0	0
58	4174	10382	824	731	0	131	0	0
59	4741	11703	1334	392	0	72	0	0
60	4375	10565	1126	624	0	86	0	0
61	4894	12001	1523	699	0	98	0	0
62	4748	12077	2093	843	0	245	0	0
63	4368	10682	1106	484	0	410	0	0
64	4517	11288	970	1420	0	59	0	0
65	4000	10679	1069	696	0	141	0	0
66	4838	13006	1066	2001	0	2300	0	0
67	4881	12519	1987	750	0	1891	0	0
68	4546	11533	1027	1520	0	66	0	0
69	4035	10892	4406	1241	0	39	0	0
70	4336	12676	1741	659	0	5252	0	0
71	3703	11028	1616	354	0	1853	0	0
72	4207	12130	1935	1632	0	1185	0	0
73	4094	11845	1159	555	0	73	0	0
74	3887	11394	1479	1035	0	1096	0	0
75	4525	13222	3241	507	0	884	0	0
76	5384	14785	1328	948	0	60	0	0
77	4002	12157	3554	1229	0	973	0	0
78	4724	13715	1328	724	0	139	0	0
79	4284	11890	930	1535	0	197	0	0
80	15683	239	1283	842	0	57	0	0
81	5254	14780	2150	544	0	331	0	0
82	4535	12278	1164	963	0	68	0	0
83	3737	10501	825	1386	0	160	0	0
84	4090	11611	657	982	0	62	0	0
85	3878	11221	1114	1186	0	496	0	0
86	4019	10761	1131	606	0	149	0	0
87	14353	366	1796	432	0	336	0	0
88	4066	11417	1947	621	0	600	0	0
89	4727	12895	1359	681	0	56	0	0
90	3697	10603	1166	4744	0	3142	0	0
91	3925	11027	1233	655	0	227	0	0
92	4021	11225	856	1408	0	155	0	0
93	3761	10652	1041	649	0	806	0	0
94	4263	12492	1261	707	0	2245	0	0
95	3885	10805	1134	577	0	100	0	0
96	4039	11443	1183	1056	0	4884	0	0
97	3655	10311	787	1859	0	1218	0	0
98	4334	10355	977	407	0	48	0	0
99	4136	10200	1124	703	0	207	0	0
100	3903	10723	785	1823	0	3761	0	0

[yk-tanigawa]

• Many reads contain a lot of mismatches.
• PacBio file does not have MAPQ and base call quality score

[yk-tanigawa]

sample screen shots on iGV for this data set

[yk-tanigawa]

filter reads

• filter by conditions:
  • mismatch rate <= 10%
  • length >= 10kb
• (in progress)

[yk-tanigawa]

pipeline

• download data
• extract reads >= 10kb
  • https://github.com/rivas-lab/nanopore/blob/20161202_pacbio_filter_script/scripts/20161201/giab.na12878.extract.sbatch
  • /share/PI/mrivas/data/giab/NA12878_PacBio_MtSinai/sorted_final_merged.10kb.bam

[yk-tanigawa]

• filter by the following condition:
  • length >- 10kb
  • mismatch ratio <= 10%
  • https://github.com/rivas-lab/nanopore/blob/20161217GIABextraction/scripts/20161216/extract-giab.sbatch
  • We have 168245 reads in /share/PI/mrivas/data/giab/NA12878_PacBio_MtSinai/sorted_final_merged.10kb.ext.sam

[yk-tanigawa]

• Check if the mismatch with high confidence (high base calling quality) against dbSNPs
  • https://github.com/rivas-lab/nanopore/blob/20161218dbsnp/scripts/20161218/dbsnp-pacbio-q14.sbatch