Challenges remain to be addressed in terms of quantifying source origins for microbiome samples in a fast, comprehensive, and context-aware manner. Traditional approaches to such quantification have severe trade-offs between efficiency, accuracy, and scalability.
Here, we introduce EXPERT, a scalable community-level microbial source tracking approach. Built upon the biome ontology information and transfer learning techniques, EXPERT has acquired the context-aware flexibility and could easily expand the supervised model's search scope to include the context-depende/nt community samples and understudied biomes. While at the same time, it is superior to current approaches in source tracking accuracy and speed. EXPERT's superiority has been demonstrated on multiple source tracking tasks, including source tracking samples collected at different disease stages and longitudinal samples. For details refer to our original study.
Supervised learning (with high efficiency and accuracy) meets transfer learning (with inherent high scalability), towards better understanding the dark matters in microbial community.
For support using EXPERT, please contact us.
This is our beta version, any comments or insights would be greatly appreciated.
tensorflow.keras
You can simply install EXPERT using pip
package manager.
pip install expert-mst # Install EXPERT
expert init # Initialize EXPERT and install NCBI taxonomy database
Here we quickly go-through basic functionalities of EXPERT through a case study, which have already been conducted in our preprinted paper. We also provided more functional show-cases in another repository.
EXPERT's fantastic function is its automatic generalization of fundamental models, which allows non-deep-learning users to modify the models just in terminal, without the need of any programming skill. Here we generalize a fundamental model for monitoring the progression of colorectal cancer (CRC) and assess the performance of the generalized model. We only use the disease model trained for quantifying contribution from hosts with different disease-associated biomes (refer to our preprint for details).
Microbial source tracking: Bayesian community-wide culture-independent microbial source tracking | Nature Methods
Cross-validation: Cross-validation (statistics) - Wikipedia
Please follow our instructions below and make sure all these commands were run on Linux/Mac OSX platform. You may also need to install Anaconda before starting.
pip install https://github.com/HUST-NingKang-Lab/EXPERT/releases/download/v0.2/expert-0.2_cpu-py3-none-any.whl
expert init
CM
is a abbreviation term of countMatrix
, which is a format of abundance data (each row represents a taxon, and each column represents a sample/run). Mapper
is another important input of EXPERT, which records source biomes for input samples.wget -c https://github.com/HUST-NingKang-Lab/EXPERT/releases/download/v0.2-m/disease_model.tgz
tar zxvf disease_model.tgz # Decompress the fundamental model.
for file in {QueryCM.tsv,SourceCM.tsv,QueryMapper.csv,SourceMapper.csv}; do
wget -c https://raw.githubusercontent.com/HUST-NingKang-Lab/EXPERT/master/data/$file;
done
grep -v "Env" SourceMapper.csv | awk -F ',' '{print $6}' | sort | uniq > microbiomes.txt
expert construct -i microbiomes.txt -o ontology.pkl
expert map --to-otlg -i SourceMapper.csv -t ontology.pkl -o SourceLabels.h5
expert map --to-otlg -i QueryMapper.csv -t ontology.pkl -o QueryLabels.h5
hdf
file. The EXPERT model only accepts standardized abundance data. Here we standardize the abundance data using convert
mode.ls SourceCM.tsv > inputList; expert convert -i inputList -o SourceCM.h5 --in-cm;
ls QueryCM.tsv > inputList; expert convert -i inputList -o QueryCM.h5 --in-cm;
rm inputList
expert transfer -i SourceCM.h5 -l SourceLabels.h5 -t ontology.pkl -m disease_model -o CRC_model
expert search -i QueryCM.h5 -m CRC_model -o quantified_source_contributions
expert evaluate -i quantified_source_contributions -l QueryLabels.h5 -o performance_report
cat performance_report/overall.csv
You now have acquired skills of EXPERT modeling for microbial source tracking. Next, you may want to explore a question: Which fundamental model gives the best performance on the CRC monitoring? You may want to assess the performance utilizing another fundamental model. Good luck.
EXPERT has enabled the adaptation to context-dependent studies, in which you can choose potential sources to be estimated. Please follow our documentation: advanced usage.
Model | Biome ontology | Top-level biome | Data source | Dataset size | Download link | Note |
---|---|---|---|---|---|---|
general model | biome ontology for 132 biomes on earth (as of Jan. 2020) | root | MGnify | 115,892 | download | The samples were not uniformly processed by MGnify |
human model | biome ontology for 27 human-associated biomes | human | MGnify | 52,537 | download | The samples were not uniformly processed by MGnify |
disease model | biome ontology for 20 human disease-associated biomes | root (human gut) | GMrepo | 13,642 | download | The samples were uniformly processed by GMrepo |
Note: These models were trained on EXPERT version 0.2.
If you are using EXPERT in a scientific publication (or inspired by the approach), we would appreciate citations to the following paper:
Hui Chong, Yuguo Zha, Qingyang Yu, Mingyue Cheng, Guangzhou Xiong, Nan Wang, Xinhe Huang, Shijuan Huang, Chuqing Sun, Sicheng Wu, Wei-Hua Chen, Luis Pedro Coelho, Kang Ning*. EXPERT: transfer learning-enabled context-aware microbial community classification. Briefings in Bioinformatics, 2022; bbac396. doi:10.1093/bib/bbac396.
Name | Organization | |
---|---|---|
Hui Chong | huichong.me@gmail.com | Research Assistant, School of Life Science and Technology, Huazhong University of Science & Technology |
Xinhe Huang | huangxinhe@hust.edu.cn | Undergraduate,School of Life Science and Technology, Huazhong University of Science & Technology |
Shijuan Huang | hshijuan@qq.com | Undergraduate,School of Life Science and Technology, Huazhong University of Science & Technology |
Kang Ning | ningkang@hust.edu.cn | Professor, School of Life Science and Technology, Huazhong University of Science & Technology |