9/16/24 Note: We are working on fixing issues in this repository, caused by changes in the underlying rPPG-Toolbox. We do not have an estimated time for the fix at the moment, but those interested in infant respiration may be able to use the rPPG-Toolbox repository directly, together with our dataset. Thanks for your interest in our work, and apologies for the inconvenience.

Automatic Infant Respiration Estimation from Video

Official repository for our paper: Automatic Infant Respiration Estimation from Video: A Deep Flow-based Algorithm and a Novel Public Benchmark, published at MICCAI PIPPI 2023.

Dataset details

We provide a new manually annotated dataset of 125 videos, Annotated Infant Respiration (AIR-125), for benchmarking respiration estimation methods on infants using videos from baby monitors and Youtube. More details on the dataset generation procedure can be found in the paper.

• AIR-125 can be downloaded from here.

• We also evaluate our method on an adult respiation datastet, COHFACE. The dataset can be downloaded from here.

• To generate optical flow inputs for our model generate dense flow outputs using Coarse2Fine optical flow implementation from here.

• To train and evaluate methods on these datasets, the dataset paths should be updated in relevant config files as detailed below.

Model

We compare our AIRFlowNet model against current physiological signal estimation models and a classical computer vision method that uses pretrained weights:

• DeepPhys: DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks

• EfficientPhys: EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Cardiac Measurement

• TS-CAN: MTTS-CAN: Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement

• Guo et. al: Remote estimation of respiration rate by optical flow using convolutional neural networks

Results

The following table from the paper compares the above methods against AIRFlowNet on COHFACE and AIR-1245 datasets. Mean absolute error (MAE), root mean squared error (RMSE), and Pearson's correlation coefficient (ρ) are used to compare the methods.

Experiments

Setup

Required packages can be installed following the below steps:

$ bash setup.sh
$ conda activate rppg-toolbox
$ pip install -r requirements.txt

Training

Use relavent config files for a specific model-train-validation-test split configuration under configs/train_configs folder.

1. Modify the dataset locations in the config file to an appropriate path.

2. Run the dataset preprocessing (by setting DO_PREPROCESS: True in the config file) when running the experiments for the first time.

3. Run python main.py --config_file ./configs/train_configs/TRAIN_VALIDATION_TEST_MODEL.yaml

Testing

Use relevant config files for a specific model-test_split configuration under configs/infer_configs folder.

1. Modify the test dataset location in the config file to an appropriate path.

2. Run python main.py --config_file ./configs/infer_configs/TRAIN_VALIDATION_TEST_MODEL.yaml

Acknowledgement

This repository is based on rPPG-Toolbox. We thank the authors for open sourcing their code.

Citation

If you find use this code or dataset for your research, please consider citing our paper:

@InProceedings{manne_2023_automatic,
author="Manne, Sai Kumar Reddy
and Zhu, Shaotong
and Ostadabbas, Sarah
and Wan, Michael",
editor="Link-Sourani, Daphna
and Abaci Turk, Esra
and Macgowan, Christopher
and Hutter, Jana
and Melbourne, Andrew
and Licandro, Roxane",
title="Automatic Infant Respiration Estimation from Video: A Deep Flow-Based Algorithm and a Novel Public Benchmark",
booktitle="Perinatal, Preterm and Paediatric Image Analysis",
year="2023",
publisher="Springer Nature Switzerland",
address="Cham",
pages="111--120",
isbn="978-3-031-45544-5"
}