CLARK-S - Githubissues

rx32940 commented 5 years ago

CLARK-S uses a spaced discriminative database for classification
taxonomic profiling using CLARK-S with Bacteria + Viruses database on genus/phylum/domain/species levels
taxonomic profiling using CLARK-S with Custom (Bacteria + Viruses + Rat genome) database on genus/phylum/domain/species levels

rx32940 commented 5 years ago

The following code should:

set genus and custom db as targets

classify R22.K sample with CLARK on genus level

/scratch/rx32940/CLARK/CLARKSCV1.2.6.1/result_genus

build spaced database with the set target by converting discrimative database to spaced discrimative database

classify R22.K sample with CLARK-S on genus level

/scratch/rx32940/CLARK/output/genus_custom_spaced

https://github.com/rx32940/Lepto-Metagenomics/blob/238eddc00ddfe555f8816001c330b7e503fa547a/run_clark.sh

rx32940 commented 5 years ago

the spaced database is building on sapelo2 with both prebuilt Clark database as well as the custom database.

set different targets for taxonomy levels and databases(custom/bacteria&viruses)
change result dir and set target command line for different bases
a total of 12 databases is building (31k discrimative database for a new taxonomy level and database combination need to built before the spaced database for the same combination)
jobs submitted to sapelo2 separately for each pair of nonspaced/spaced database building
code: https://github.com/rx32940/Lepto-Metagenomics/blob/23e8c7fb20b244ea1197f535f09d345aa11b2da8/run_clark.sh

Results for prebuilt regular database:

/scratch/rx32940/CLARK/output/prebuilt/genus(phylum/species)/regular

Results for prebuilt spaced database:

/scratch/rx32940/CLARK/output/prebuilt/genus(phylum/species)/spaced

Results for custom regular database:

/scratch/rx32940/CLARK/output/custom/genus(phylum/species)/regular

Results for custom spaced database:

/scratch/rx32940/CLARK/output/custom/genus(phylum/species)/spaced

[ ] When database building is done, classify each sample with each combination of databases for each sample separately.

rx32940 commented 5 years ago

[x] Phylum level analysis (regular and spaced with custom database)
[x] r visualization barplot fro phylum level analysis
[ ] genus level analysis (regular and spaced with custom database)
[ ] top 10 most abundant genus
[ ] species level analysis (regular and spaced with custom database)

rx32940 commented 5 years ago

[x] Phylum level analysis (regular and spaced with custom database)

[x] r visualization barplot fro phylum level analysis

[ ] genus level analysis (regular and spaced with custom database)

[ ] top 10 most abundant genus

[ ] species level analysis (regular and spaced with custom database)

Clark-s(23) and Clark classified a lot fewer phylum taxa with custom database compare to Clark classified with prebuilt databases (43). only 22 taxa in total were classified with the 12 samples
- possible explanation: with the addition of the host refseq into the database, many of the ambiguous reads is now classified as Rattus
compare number of phylum taxa identified for
[x] Clark-s custom database (23)

Clark-s custom database results

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark-s/phylum/classifiedOnly_Count_clarks_phylum_custom.csv

[x] Clark custom database (36)

Clark custom database results

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark/phylum/results/classifiedOnly_percentage_clark_phylum_custom.csv

[x] Clark-s prebuilt database (26)

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark-s/phylum/prebuilt/classifiedOnly_Count_clark_phylum_custom.csv

Clark-s prebuilt database results

[x] Clark prebuilt database (41)

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark/phylum/prebuilt/classifiedOnly_count_clark_phylum_prebuilt.csv

[x] kraken2 custom database (31) r visualization of the kraken2/bracken phylum level classification with custom database

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Kraken2-standard/custom/phylum/bracken_%_custom_phylum.csv

[x] kraken2 standard database kraken2 standard database results
thus all phylum classified by Clark-s with custom database were plotted here.
- for comparison with Clark results classified with prebuilt database, please see here: Clark w/prebuilt database

Sample ID	Classified (%)	Unclassified (%)
R22.K	82.4363	17.5637
R22.L	28.7164	71.2836
R22.S	73.2934	26.7066
R26.K	77.8431	22.1569
R26.L	48.262	51.738
R26.S	72.1112	27.8888
R27.K	72.597	27.403
R27.L	31.2137	68.7863
R27.S	71.7787	28.2213
R28.K	90.76	9.24
R28.L	88.6129	11.3871
R28.S	89.0807	10.9193

dir for the table above:

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark-s/phylum/clarks_custom_phylum_%_classified.xlsx

w/o unclassified reads

code to plot graphs above

dir with the exact number of classified reads and percentage of the reads excluding the UNKNOWN reads:

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark-s/phylum/classifiedOnly_Count_clarks_phylum_custom.csv

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark-s/phylum/classifiedOnly_percentage_clarks_phylum_custom.csv

rx32940 commented 5 years ago

classification with a prebuilt database shows a very low classification level, thus will not be considered.

CLARK-s prebuilt database phylum level classification

Sample ID	Classified (%)	Unclassified (%)
R22.K	10.4762	89.5238
R22.L	6.2897	93.7103
R22.S	12.3613	87.6387
R26.K	12.786	87.214
R26.L	7.2184	92.7816
R26.S	8.9735	91.0265
R27.K	15.6412	84.3588
R27.L	6.2805	93.7195
R27.S	8.3282	91.6718
R28.K	6.7079	93.2921
R28.L	5.1649	94.8351
R28.S	4.5048	95.4952

a lot less reads were classified. Those reads probably belong to the host (Rattus) genome

rx32940 commented 5 years ago

From the percent of classified reads file, I concluded that:

with a custom database, Clark-s was able to classify more reads compare to Clark.
However, with a prebuilt database. the percentage of reads Clark-s was able to classify was less than that of Clark.
with this observation, I think Clark-s probably has a better accuracy for classification by not classify reads into taxa that it does not belong to.

dir to the classified percentage file:

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark-s/phylum/clarks_custom_phylum_%_classified.xlsx

rx32940 commented 5 years ago

Clark Custom Database Phylum Level Classification

Sample ID	Classified (%)	Unclassified (%)
R22.K	76.8498	23.1502
R22.L	25.789	74.211
R22.S	66.2754	33.7246
R26.K	73.5044	26.4956
R26.L	42.8965	57.1035
R26.S	64.2384	35.7616
R27.K	70.1686	29.8314
R27.L	27.828	72.172
R27.S	64.6651	35.3349
R28.K	80.2579	19.7421
R28.L	78.6874	21.3126
R28.S	79.2095	20.7905

clark_phylum_custom_relative clark_phylum_custom_absolute

rx32940 commented 5 years ago

Clark-s genus results with custom database

dir for the results:

/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark-s/genus/results

Code for genus level clark-s classification r visualization clark-s_genus_custom_Relative clark-s_genus_custom_absolute

rx32940 commented 5 years ago

Clark genus results with custom database

in Clark results, large proportion of reads were classified as plantactinspora while this taxa was never identified in these samples with Clark-s dir for the results:
```
/Users/rx32940/Dropbox/5.Rachel-projects/Metagenomic_Analysis/Clark/genus/results
```

rx32940 commented 5 years ago

Kraken2+Bracken genus level classification

https://github.com/rx32940/Lepto-Metagenomics/issues/3#issuecomment-547548115

rx32940 / Lepto-Metagenomics

CLARK-S #4

CLARK-s prebuilt database phylum level classification

Clark Custom Database Phylum Level Classification

Clark-s genus results with custom database

Clark genus results with custom database

Kraken2+Bracken genus level classification