search

sunlabuiuc / PyHealth

A Deep Learning Python Toolkit for Healthcare Applications.
https://pyhealth.readthedocs.io
MIT License
994 stars 212 forks source link
clinical-data clinical-research data-mining deep-learning electronic-health-record electronic-medical-record healthcare medical-code preprocessing

readme

Welcome to PyHealth!

.. image:: https://img.shields.io/pypi/v/pyhealth.svg?color=brightgreen :target: https://pypi.org/project/pyhealth/ :alt: PyPI version

.. image:: https://readthedocs.org/projects/pyhealth/badge/?version=latest :target: https://pyhealth.readthedocs.io/en/latest/ :alt: Documentation status

.. image:: https://img.shields.io/github/stars/sunlabuiuc/pyhealth.svg :target: https://github.com/sunlabuiuc/pyhealth/stargazers :alt: GitHub stars

.. image:: https://img.shields.io/github/forks/sunlabuiuc/pyhealth.svg?color=blue :target: https://github.com/sunlabuiuc/pyhealth/network :alt: GitHub forks

.. image:: https://static.pepy.tech/badge/pyhealth :target: https://pepy.tech/project/pyhealth :alt: Downloads

.. image:: https://img.shields.io/badge/Tutorials-Google%20Colab-red :target: https://pyhealth.readthedocs.io/en/latest/tutorials.html :alt: Tutorials

.. image:: https://img.shields.io/badge/YouTube-16%20Videos-red :target: https://www.youtube.com/playlist?list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV :alt: YouTube

.. -----

.. Build Status & Coverage & Maintainability & License

.. .. image:: https://travis-ci.org/yzhao062/pyhealth.svg?branch=master .. :target: https://travis-ci.org/yzhao062/pyhealth .. :alt: Build Status

.. .. image:: https://ci.appveyor.com/api/projects/status/1kupdy87etks5n3r/branch/master?svg=true .. :target: https://ci.appveyor.com/project/yzhao062/pyhealth/branch/master .. :alt: Build status

.. .. image:: https://api.codeclimate.com/v1/badges/bdc3d8d0454274c753c4/maintainability .. :target: https://codeclimate.com/github/yzhao062/pyhealth/maintainability .. :alt: Maintainability

.. .. image:: https://img.shields.io/github/license/yzhao062/pyhealth .. :target: https://github.com/yzhao062/pyhealth/blob/master/LICENSE .. :alt: License

Citing PyHealth :handshake:

Yang, Chaoqi, Zhenbang Wu, Patrick Jiang, Zhen Lin, Junyi Gao, Benjamin P. Danek, and Jimeng Sun. 2023. “PyHealth: A Deep Learning Toolkit for Healthcare Applications.” In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 5788–89. KDD ’23. New York, NY, USA: Association for Computing Machinery.

.. code-block:: bibtex

@inproceedings{pyhealth2023yang,
    author = {Yang, Chaoqi and Wu, Zhenbang and Jiang, Patrick and Lin, Zhen and Gao, Junyi and Danek, Benjamin and Sun, Jimeng},
    title = {{PyHealth}: A Deep Learning Toolkit for Healthcare Predictive Modeling},
    url = {https://github.com/sunlabuiuc/PyHealth},
    booktitle = {Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) 2023},
    year = {2023}
}

Checkout Our KDD'23 Tutorial https://sunlabuiuc.github.io/PyHealth/

PyHealth is a comprehensive deep learning toolkit for supporting clinical predictive modeling, which is designed for both ML researchers and medical practitioners. We can make your healthcare AI applications easier to deploy and more flexible and customizable. [Tutorials] <https://pyhealth.readthedocs.io/>_

[News!] We are continueously implemeting good papers and benchmarks into PyHealth, checkout the [planned List] <https://docs.google.com/spreadsheets/d/1PNMgDe-llOm1SM5ZyGLkmPysjC4wwaVblPLAHLxejTw/edit#gid=159213380>_. Welcome to pick one from the list and send us a PR or add more influential and new papers into the plan list.

.. image:: figure/poster.png :width: 810

..

  1. Installation :rocket:

.. code-block:: sh

pip install pyhealth

.. code-block:: sh

pip install .

  1. Introduction :book:

    pyhealth provides these functionalities (we are still enriching some modules):

.. image:: figure/overview.png :width: 770

You can use the following functions independently:

Building a healthcare AI pipeline can be as short as 10 lines of code in PyHealth.

  1. Build ML Pipelines :trophy:

All healthcare tasks in our package follow a five-stage pipeline:

.. image:: figure/five-stage-pipeline.png :width: 640

..

We try hard to make sure each stage is as separate as possible, so that people can customize their own pipeline by only using our data processing steps or the ML models.

Module 1: """"""""""""""""""""""""""""""""""""

pyhealth.datasets provides a clean structure for the dataset, independent from the tasks. We support MIMIC-III, MIMIC-IV and eICU, etc. The output (mimic3base) is a multi-level dictionary structure (see illustration below).

.. code-block:: python

from pyhealth.datasets import MIMIC3Dataset

mimic3base = MIMIC3Dataset(
    # root directory of the dataset
    root="https://storage.googleapis.com/pyhealth/Synthetic_MIMIC-III/", 
    # raw CSV table name
    tables=["DIAGNOSES_ICD", "PROCEDURES_ICD", "PRESCRIPTIONS"],
    # map all NDC codes to CCS codes in these tables
    code_mapping={"NDC": "CCSCM"},
)

.. image:: figure/structured-dataset.png :width: 400

..

Module 2: """"""""""""""""""""""""""""""""""""

pyhealth.tasks defines how to process each patient's data into a set of samples for the tasks. In the package, we provide several task examples, such as drug recommendation and length of stay prediction. It is easy to customize your own tasks following our template <https://colab.research.google.com/drive/1r7MYQR_5yCJGpK_9I9-A10HmpupZuIN-?usp=sharing>_.

.. code-block:: python

from pyhealth.tasks import readmission_prediction_mimic3_fn

mimic3sample = mimic3base.set_task(task_fn=readmission_prediction_mimic3_fn) # use default task
mimic3sample.samples[0] # show the information of the first sample
"""
{
    'visit_id': '100183',
    'patient_id': '175',
    'conditions': ['5990', '4280', '2851', '4240', '2749', '9982', 'E8499', '42831', '34600'],
    'procedures': ['0040', '3931', '7769'],
    'drugs': ['N06DA02', 'V06DC01', 'B01AB01', 'A06AA02', 'R03AC02', 'H03AA01', 'J01FA09'],
    'label': 0
}
"""

from pyhealth.datasets import split_by_patient, get_dataloader

train_ds, val_ds, test_ds = split_by_patient(mimic3sample, [0.8, 0.1, 0.1])
train_loader = get_dataloader(train_ds, batch_size=32, shuffle=True)
val_loader = get_dataloader(val_ds, batch_size=32, shuffle=False)
test_loader = get_dataloader(test_ds, batch_size=32, shuffle=False)

Module 3: """"""""""""""""""""""""""""""""""""

pyhealth.models provides different ML models with very similar argument configs.

.. code-block:: python

from pyhealth.models import Transformer

model = Transformer(
    dataset=mimic3sample,
    feature_keys=["conditions", "procedures", "drug"],
    label_key="label",
    mode="binary",
)

Module 4: """"""""""""""""""""""""""""""""""""

pyhealth.trainer can specify training arguments, such as epochs, optimizer, learning rate, etc. The trainer will automatically save the best model and output the path in the end.

.. code-block:: python

from pyhealth.trainer import Trainer

trainer = Trainer(model=model)
trainer.train(
    train_dataloader=train_loader,
    val_dataloader=val_loader,
    epochs=50,
    monitor="pr_auc_samples",
)

Module 5: """"""""""""""""""""""""""""""""""""

pyhealth.metrics provides several common evaluation metrics (refer to Doc <https://pyhealth.readthedocs.io/en/latest/api/metrics.html>_ and see what are available).

.. code-block:: python

# method 1
trainer.evaluate(test_loader)

# method 2
from pyhealth.metrics.binary import binary_metrics_fn

y_true, y_prob, loss = trainer.inference(test_loader)
binary_metrics_fn(y_true, y_prob, metrics=["pr_auc", "roc_auc"])

  1. Medical Code Map :hospital:

pyhealth.codemap provides two core functionalities. This module can be used independently.

.. code-block:: python

from pyhealth.medcode import InnerMap

icd9cm = InnerMap.load("ICD9CM")
icd9cm.lookup("428.0")
# `Congestive heart failure, unspecified`
icd9cm.get_ancestors("428.0")
# ['428', '420-429.99', '390-459.99', '001-999.99']

atc = InnerMap.load("ATC")
atc.lookup("M01AE51")
# `ibuprofen, combinations`
atc.lookup("M01AE51", "drugbank_id")
# `DB01050`
atc.lookup("M01AE51", "description")
# Ibuprofen is a non-steroidal anti-inflammatory drug (NSAID) derived ...
atc.lookup("M01AE51", "indication")
# Ibuprofen is the most commonly used and prescribed NSAID. It is very common over the ...

.. code-block:: python

from pyhealth.medcode import CrossMap

codemap = CrossMap.load("ICD9CM", "CCSCM")
codemap.map("428.0")
# ['108']

codemap = CrossMap.load("NDC", "RxNorm")
codemap.map("50580049698")
# ['209387']

codemap = CrossMap.load("NDC", "ATC")
codemap.map("50090539100")
# ['A10AC04', 'A10AD04', 'A10AB04']

  1. Medical Code Tokenizer :speech_balloon:

pyhealth.tokenizer is used for transformations between string-based tokens and integer-based indices, based on the overall token space. We provide flexible functions to tokenize 1D, 2D and 3D lists. This module can be used independently.

.. code-block:: python

from pyhealth.tokenizer import Tokenizer

# Example: we use a list of ATC3 code as the token
token_space = ['A01A', 'A02A', 'A02B', 'A02X', 'A03A', 'A03B', 'A03C', 'A03D', \
        'A03F', 'A04A', 'A05A', 'A05B', 'A05C', 'A06A', 'A07A', 'A07B', 'A07C', \
        'A12B', 'A12C', 'A13A', 'A14A', 'A14B', 'A16A']
tokenizer = Tokenizer(tokens=token_space, special_tokens=["<pad>", "<unk>"])

# 2d encode 
tokens = [['A03C', 'A03D', 'A03E', 'A03F'], ['A04A', 'B035', 'C129']]
indices = tokenizer.batch_encode_2d(tokens) 
# [[8, 9, 10, 11], [12, 1, 1, 0]]

# 2d decode 
indices = [[8, 9, 10, 11], [12, 1, 1, 0]]
tokens = tokenizer.batch_decode_2d(indices)
# [['A03C', 'A03D', 'A03E', 'A03F'], ['A04A', '<unk>', '<unk>']]

# 3d encode
tokens = [[['A03C', 'A03D', 'A03E', 'A03F'], ['A08A', 'A09A']], \
    [['A04A', 'B035', 'C129']]]
indices = tokenizer.batch_encode_3d(tokens)
# [[[8, 9, 10, 11], [24, 25, 0, 0]], [[12, 1, 1, 0], [0, 0, 0, 0]]]

# 3d decode
indices = [[[8, 9, 10, 11], [24, 25, 0, 0]], \
    [[12, 1, 1, 0], [0, 0, 0, 0]]]
tokens = tokenizer.batch_decode_3d(indices)
# [[['A03C', 'A03D', 'A03E', 'A03F'], ['A08A', 'A09A']], [['A04A', '<unk>', '<unk>']]]

..

  1. Tutorials :teacher:

.. image:: https://colab.research.google.com/assets/colab-badge.svg :target: https://pyhealth.readthedocs.io/en/latest/tutorials.html

..

We provide the following tutorials to help users get started with our pyhealth.

Tutorial 0: Introduction to pyhealth.data <https://colab.research.google.com/drive/1y9PawgSbyMbSSMw1dpfwtooH7qzOEYdN?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=Nk1itBoLOX8&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=2>__

Tutorial 1: Introduction to pyhealth.datasets <https://colab.research.google.com/drive/18kbzEQAj1FMs_J9rTGX8eCoxnWdx4Ltn?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=c1InKqFJbsI&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=3>__

Tutorial 2: Introduction to pyhealth.tasks <https://colab.research.google.com/drive/1r7MYQR_5yCJGpK_9I9-A10HmpupZuIN-?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=CxESe1gYWU4&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=4>__

Tutorial 3: Introduction to pyhealth.models <https://colab.research.google.com/drive/1LcXZlu7ZUuqepf269X3FhXuhHeRvaJX5?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=fRc0ncbTgZA&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=6>__

Tutorial 4: Introduction to pyhealth.trainer <https://colab.research.google.com/drive/1L1Nz76cRNB7wTp5Pz_4Vp4N2eRZ9R6xl?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=5Hyw3of5pO4&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=7>__

Tutorial 5: Introduction to pyhealth.metrics <https://colab.research.google.com/drive/1Mrs77EJ92HwMgDaElJ_CBXbi4iABZBeo?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=d-Kx_xCwre4&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=8>__

Tutorial 6: Introduction to pyhealth.tokenizer <https://colab.research.google.com/drive/1bDOb0A5g0umBjtz8NIp4wqye7taJ03D0?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=CeXJtf0lfs0&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=10>__

Tutorial 7: Introduction to pyhealth.medcode <https://colab.research.google.com/drive/1xrp_ACM2_Hg5Wxzj0SKKKgZfMY0WwEj3?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=MmmfU6_xkYg&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=9>__

The following tutorials will help users build their own task pipelines.

Pipeline 1: Drug Recommendation <https://colab.research.google.com/drive/10CSb4F4llYJvv42yTUiRmvSZdoEsbmFF?usp=sharing>_ [Video] <https:// www.youtube.com/watch?v=GGP3Dhfyisc&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=12>__

Pipeline 2: Length of Stay Prediction <https://colab.research.google.com/drive/1JoPpXqqB1_lGF1XscBOsDHMLtgvlOYI1?usp=sharing>_ [Video] <https:// www.youtube.com/watch?v=GGP3Dhfyisc&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=12>__

Pipeline 3: Readmission Prediction <https://colab.research.google.com/drive/1bhCwbXce1YFtVaQLsOt4FcyZJ1_my7Cs?usp=sharing>_ [Video] <https:// www.youtube.com/watch?v=GGP3Dhfyisc&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=12>__

Pipeline 4: Mortality Prediction <https://colab.research.google.com/drive/1Qblpcv4NWjrnADT66TjBcNwOe8x6wU4c?usp=sharing>_ [Video] <https:// www.youtube.com/watch?v=GGP3Dhfyisc&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=12>__

Pipeline 5: Sleep Staging <https://colab.research.google.com/drive/1mpSeNCAthXG3cqROkdUcUdozIPIMTCuo?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=ySAIU-rO6so&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=16>__

We provided the advanced tutorials for supporting various needs.

Advanced Tutorial 1: Fit your dataset into our pipeline <https://colab.research.google.com/drive/1UurxwAAov1bL_5OO3gQJ4gAa_paeJwJp?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=xw2hGLEQ4Y0&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=13>__

Advanced Tutorial 2: Define your own healthcare task <https://colab.research.google.com/drive/1gK6zPXvfFGBM1uNaLP32BOKrnnJdqRq2?usp=sharing>_

Advanced Tutorial 3: Adopt customized model into pyhealth <https://colab.research.google.com/drive/1F_NJ90GC8_Eq-vKTf7Tyziew4gWjjKoH?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=lADFlcmLtdE&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=14>__

Advanced Tutorial 4: Load your own processed data into pyhealth and try out our ML models <https://colab.research.google.com/drive/1ZRnKch2EyJLrI3G5AvDXVpeE2wwgBWfw?usp=sharing>_ [Video] <https://www.youtube.com/watch?v=xw2hGLEQ4Y0&list=PLR3CNIF8DDHJUl8RLhyOVpX_kT4bxulEV&index=13>__

  1. Datasets :mountain_snow:

    We provide the processing files for the following open EHR datasets:

=================== ======================================= ======================================== ======================================================================================================== Dataset Module Year Information
=================== ======================================= ======================================== ======================================================================================================== MIMIC-III pyhealth.datasets.MIMIC3Dataset 2016 MIMIC-III Clinical Database <https://physionet.org/content/mimiciii/1.4//>
MIMIC-IV pyhealth.datasets.MIMIC4Dataset 2020 MIMIC-IV Clinical Database <https://physionet.org/content/mimiciv/0.4/>
eICU pyhealth.datasets.eICUDataset 2018 eICU Collaborative Research Database <https://eicu-crd.mit.edu//>
OMOP pyhealth.datasets.OMOPDataset OMOP-CDM schema based dataset <https://www.ohdsi.org/data-standardization/the-common-data-model/>
SleepEDF pyhealth.datasets.SleepEDFDataset 2018 Sleep-EDF dataset <https://physionet.org/content/sleep-edfx/1.0.0/> SHHS pyhealth.datasets.SHHSDataset 2016 Sleep Heart Health Study dataset <https://sleepdata.org/datasets/shhs>
ISRUC pyhealth.datasets.ISRUCDataset 2016 ISRUC-SLEEP dataset <https://sleeptight.isr.uc.pt/?page_id=48>_
=================== ======================================= ======================================== ========================================================================================================

  1. Machine/Deep Learning Models and Benchmarks :airplane:

================================== ================ ================================= ====== ============================================================================================================================================================================ ======================================================================================================================================================================================= Model Name Type Module Year Summary Reference ================================== ================ ================================= ====== ============================================================================================================================================================================ ======================================================================================================================================================================================= Multi-layer Perceptron deep learning pyhealth.models.MLP 1986 MLP treats each feature as static Backpropagation: theory, architectures, and applications <https://www.taylorfrancis.com/books/mono/10.4324/9780203763247/backpropagation-yves-chauvin-david-rumelhart> Convolutional Neural Network (CNN) deep learning pyhealth.models.CNN 1989 CNN runs on the conceptual patient-by-visit grids Handwritten Digit Recognition with a Back-Propagation Network <https://proceedings.neurips.cc/paper/1989/file/53c3bce66e43be4f209556518c2fcb54-Paper.pdf> Recurrent Neural Nets (RNN) deep Learning pyhealth.models.RNN 2011 RNN (includes LSTM and GRU) can run on any sequential level (e.g., visit by visit sequences) Recurrent neural network based language model <http://www.fit.vutbr.cz/research/groups/speech/servite/2010/rnnlm_mikolov.pdf> Transformer deep Learning pyhealth.models.Transformer 2017 Transformer can run on any sequential level (e.g., visit by visit sequences) Attention Is All You Need <https://arxiv.org/abs/1706.03762> RETAIN deep Learning pyhealth.models.RETAIN 2016 RETAIN uses two RNN to learn patient embeddings while providing feature-level and visit-level importance. RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism <https://arxiv.org/abs/1608.05745> GAMENet deep Learning pyhealth.models.GAMENet 2019 GAMENet uses memory networks, used only for drug recommendation task GAMENet: Graph Augmented MEmory Networks for Recommending Medication Combination <https://arxiv.org/abs/1809.01852> MICRON deep Learning pyhealth.models.MICRON 2021 MICRON predicts the future drug combination by instead predicting the changes w.r.t. the current combination, used only for drug recommendation task Change Matters: Medication Change Prediction with Recurrent Residual Networks <https://www.ijcai.org/proceedings/2021/0513> SafeDrug deep Learning pyhealth.models.SafeDrug 2021 SafeDrug encodes drug molecule structures by graph neural networks, used only for drug recommendation task SafeDrug: Dual Molecular Graph Encoders for Recommending Effective and Safe Drug Combinations <https://arxiv.org/abs/2105.02711> MoleRec deep Learning pyhealth.models.MoleRec 2023 MoleRec encodes drug molecule in a substructure level as well as the patient's information into a drug combination representation, used only for drug recommendation task MoleRec: Combinatorial Drug Recommendation with Substructure-Aware Molecular Representation Learning <https://dl.acm.org/doi/10.1145/3543507.3583872> Deepr deep Learning pyhealth.models.Deepr 2017 Deepr is based on 1D CNN. General purpose. Deepr : A Convolutional Net for Medical Records <https://arxiv.org/abs/1607.07519> ContraWR Encoder (STFT+CNN) deep Learning pyhealth.models.ContraWR 2021 ContraWR encoder uses short time Fourier transform (STFT) + 2D CNN, used for biosignal learning Self-supervised EEG Representation Learning for Automatic Sleep Staging <https://arxiv.org/abs/2110.15278> SparcNet (1D CNN) deep Learning pyhealth.models.SparcNet 2023 SparcNet is based on 1D CNN, used for biosignal learning Development of Expert-level Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10136013/> TCN deep learning pyhealth.models.TCN 2018 TCN is based on dilated 1D CNN. General purpose An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling <https://arxiv.org/abs/1803.01271> AdaCare deep learning pyhealth.models.AdaCare 2020 AdaCare uses CNNs with dilated filters to learn enriched patient embedding. It uses feature calibration module to provide the feature-level and visit-level interpretability AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration <https://arxiv.org/abs/1911.12205> ConCare deep learning pyhealth.models.ConCare 2020 ConCare uses transformers to learn patient embedding and calculate inter-feature correlations. ConCare: Personalized Clinical Feature Embedding via Capturing the Healthcare Context <https://arxiv.org/abs/1911.12216> StageNet deep learning pyhealth.models.StageNet 2020 StageNet uses stage-aware LSTM to conduct clinical predictive tasks while learning patient disease progression stage change unsupervisedly StageNet: Stage-Aware Neural Networks for Health Risk Prediction <https://arxiv.org/abs/2001.10054> Dr. Agent deep learning pyhealth.models.Agent 2020 Dr. Agent uses two reinforcement learning agents to learn patient embeddings by mimicking clinical second opinions Dr. Agent: Clinical predictive model via mimicked second opinions <https://academic.oup.com/jamia/article/27/7/1084/5858308> GRASP deep learning pyhealth.models.GRASP 2021 GRASP uses graph neural network to identify latent patient clusters and uses the clustering information to learn patient GRASP: Generic Framework for Health Status Representation Learning Based on Incorporating Knowledge from Similar Patients <https://ojs.aaai.org/index.php/AAAI/article/view/16152> ================================== ================ ================================= ====== ============================================================================================================================================================================ =======================================================================================================================================================================================