We propose a Two-stage Residual-based Motion Deblurring (TRMD) framework for an event camera, which converts a blurry image into a sequence of sharp images, leveraging the abundant motion features encoded in events. In the first stage, a residual estimation network is trained to estimate the residual sequence, which measures the intensity difference between the intermediate frame and other frames sampled during the exposure. In the subsequent stage, the previously estimated residuals are combined with the blurry image to reconstruct the deblurred sequence based on the physical model of motion blur.
git clone https://github.com/chenkang455/TRMD
cd TRMD
pip install -r requirements.txt
You can download our trained models, synthesized dataset GOPRO and real event dataset REBlur (from EFNet) from Baidu Netdisk with the password eluc
.
Unzip the GOPRO.zip
file before placing the downloaded models and datasets (path defined in config.yaml) according to the following directory structure:
βββ Data
βΒ Β βββ GOPRO
βΒ Β βΒ Β βββ train
βΒ Β βΒ βββ test
| βββ REBlur
| | βββ train
| | βββ test
| | βββ addition
| | βββ README.md
βββ Pretrained_Model
βΒ Β βββ RE_Net.pth
βΒ Β βββ RE_Net_rgb.pth
βββ config.yaml
βββ ...
Change the data path and other parameters (if needed) in config.yaml.
python test_GoPro.py --rgb False --load_path Pretrained_Model/RE_Net_GRAY.pth
python test_GoPro.py --rgb True --load_path Pretrained_Model/RE_Net_RGB.pth
python test_REBlur.py --load_path Pretrained_Model/RE_Net_GRAY.pth
python network.py
python train_GoPro.py --rgb False --save_path Model/RE_Net_GRAY.pth
python train_GoPro.py --rgb True --save_path Model/RE_Net_RGB.pth
Should you have any questions, please feel free to contact mrchenkang@whu.edu.cn or ly.wd@whu.edu.cn.
If you find our work useful in your research, please cite:
@article{chen2024motion,
title={Motion Deblur by Learning Residual from Events},
author={Chen, Kang and Yu, Lei},
journal={IEEE Transactions on Multimedia},
year={2024},
publisher={IEEE}
}
Our event representation (SCER) code and REBlur dataset are derived from EFNet. Some of the code for metric testing and module construction is from E-CIR. We appreciate the effort of the contributors to these repositories.