Industrial Scene Text Detection with Refined Feature-attentive Network

This is the code of "Industrial Scene Text Detection with Refined Feature-attentive Network". For more details, please refer to our TCSVT paper or Poster.

Environments

Ubuntu 16.04
Cuda 10
python >=3.5
pytorch 1.0
Other packages like cv2, Polygon3, tensorboardX, Scipy.

Highlights

Training and evaluation checked: Testing in MPSC test set with training data in {SynthMPSC, MPSC}. Other scene text datasets are test with pre-training data in SynthText.
Dataset link:
- MPSC&SynthMPSC: Please refer to IndustrialTextDataset.md for dataset download.
- Synthtext
- MSRA-TD500
- ICDAR2013
- ICDAR2017-MLT
- USTB-SV1K
  Installation
  
  Check INSTALL.md for installation instructions.

Configuring your dataset

Update datset root path in$RFN_ROOT/train.py.
Process dataset can be set in $RFN_ROOT/tools/datagen.py.
Modify test path in $RFN_ROOT/multi_image_test_ocr.py.

Modify some settings in $RFN_ROOT/tools/encoder.py, including anchor_areas, aspect_ratios.

# refer to /data_process/Compute aspect_ratios and area_ratios.py
For example the setting of MPSC as follows:
self.anchor_areas = [16*16., 32*32., 64*64., 128*128., 256*256, 512*512.]
self.aspect_ratios = [1., 2., 3., 5., 1./2., 1./3., 1./5.,7.]

Training

# create your data cache directory
cd RFN_ROOT
# Download pretrained ResNet50 model(https://data.lip6.fr/cadene/pretrainedmodels/se_resnet50-ce0d4300.pth)
# Init RFN with pretrained ResNet50 model
python ./tools/get_state_dict.py
python train.py --config_file=./configs/R_50_C4_1x_train.yaml

The training size is set to a multiple of 128.
Multi-GPU phase is not testing yet, be careful to use GPU more than 1.

Test and eval

Our provide script: $RFN_ROOT/multi_image_test_ocr.py and $RFN_ROOT/test/
Modify path settings and choose the dataset you want to evaluate on.

option parameters: save_img, show_mask

### test each image
python test.py --dataset=MPSC --config_file=./configs/R_50_C4_1x_train.yaml --test
### eval result
python test.py --dataset=MPSC --eval

Pretrained Weights for Training and Testing

Here we provide some pretained weights for testing:

Pretrain SynthMPSC : https://pan.baidu.com/s/1BI2T4ncowKu908dcd9tT7g (0ke0)
Pretrain SynthText : https://pan.baidu.com/s/1IwALX0LrQewsk9Rf5cK1Dw (6dzr)

More Results

Model | Dataset | Precision | Recall | F-Measure | MODEL link | Extraction code
RFN | MPSC | 89.30 | 83.33 | 86.21 | model | 6u6y
RFN* | MPSC | 89.82 | 84.45 | 87.05 | model | xrni

Visualizations of MPSC dataset

examples1

Visualizations of MSRA-TD500, USTB-SV1K, ICDAR2013, ICDAR2017-MLT dataset

examples2

Citation

If you find our method useful for your reserach, please cite

@ARTICLE{9726175,
  author={Guan, Tongkun and Gu, Chaochen and Lu, Changsheng and Tu, Jingzheng and Feng, Qi and Wu, Kaijie and Guan, Xinping},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={Industrial Scene Text Detection With Refined Feature-Attentive Network}, 
  year={2022},
  volume={32},
  number={9},
  pages={6073-6085},
  doi={10.1109/TCSVT.2022.3156390}}

License

- This code are only free for academic research purposes and licensed under the 2-clause BSD License - see the LICENSE file for details.

TongkunGuan / RFN

readme