EUG

This repo contains the supported code and configuration files to reproduce anomaly detection results of EUG. Its code is based on mmsegmentation and jsrnet.

1. Installation

In order to run this project normally, you first need to install the dependencies required by jsrnet

pip install requirements.txt

Then you need to install mmsegmentation as instructed by the official documentation

2. Data Preparation

2.1 Training set

The data sets used in this project include:

Cityscapes (Training, 2975 images)
LostAndFound (Train set for validation, 1036 images)

2.2 Testing sets

LostAndFound (test 1203 images)
Road Anomaly (val. 60 images)
Road Obstacle (val. 30 images)
Baidu link \ Google link

NOTE: The link of testing sets is provided, and we have made following changes: 1) Convert all the *.png* files to the .jpg** images; 2) Convert images in LaF and RO to smaller sizes (1/2w × 1/2H).

2.3 Ground truth of testing set

For the testing set, the ground truth of RA is generated by the authors; for LaF and RO, the GTs are similar to the official ones.

Baidu link \ Google link

3. Model Training

3.1 Set up the path of dataset for training

Modify the path of each dataset in mypath.py

3.2 Download base model's checkpoints and configs (provided by mmsegmentation)

The selection of EUG's base model is as follows:

Methods	Base Model 1	Base Model 2
DeepEnsemble	PSP_s	PSP_s2
EUG_tiny	PSP_s	OCR_s
EUG_base	PSP	OCR
EUG_heter	OCR	Segformer

The model we chose to use is detailed below,you need to go to the official repo of [mmsegmentation]() to download the corresponding config and checkpoint files

PSP_s : pspnet_r18-d8_512x1024_80k_cityscapes
OCR_s : ocrnet_hr18s_512x1024_40k_cityscapes
PSP : pspnet_r101-d8_512x1024_40k_cityscapes
OCR ocrnet_hr48_512x1024_40k_cityscapes
Segformer segformer_mit-b2_8x1_1024x1024_160k_cityscapes
PSP_s2 ([checkpoint]())

NOTE: PSP_s2 is trained by us.

3.3 Set up the path of checkpoints and configs

To start training the model, first you need to modify the weights and configuration file paths for the base model in the configuration file. The location of the configuration file is '/exp_config/defaults.py', and the configuration items that need to be modified are as follows:

_C.EXPERIMENT.CONFIG_FILE1='xxx'
_C.EXPERIMENT.CONFIG_FILE2='xxx'
_C.EXPERIMENT.CHECKPOINT_FILE1='xxx'
_C.EXPERIMENT.CHECKPOINT_FILE2='xxx'

3.4 Training

# for EUG tiny
python train.py --model_name eug_tiny

# for EUG base
python train.py --model_name eug_base

# for EUG heter
python train.py --model_name eug_heter

4 Model Inference

4.1 Prepare Checkpoints

Model evaluation can be applied using the pretrained checkpoints or the ones trained by yourself:

NOTE: Only the fusion model weight provided.

4.2 Prepare Datasets

Download the required test dataset(The link is in section Data Preparation)

LaF
Road Anomaly
Road Obstacle

4.3 Testing

Execute the following code to perform inference for the model.

python inference.py --ckpt_path --out_dir --img_dir

'ckpt_path' is the storage path for the fuse_decoder's weights, 'out_dir' is the storage location for inference results, and 'img_dir' is the storage location for test images.

NOTE: The generated color images are for visualization and the grey scale images can be used for evaluation.

5. Model Evaluation

5.1 Prepare GTs

Prepare the GT information: all the GT information can be downloaded from following link(Generated by the authors and the links are provided in 2.3):

Laf
Road Anomaly
Road Obstacle
5.2 Set up the path

Update the path list in each subfolder to your local path, take the RA dataset for example, you should change the path list in 'path_of_GT\ra_lab\lablist.txt'.

5.3 Evaluation

Run the evaluation code to generate the results (AP,FPRs): you should change the GT path and output path in the code.
```
python eval.py
```

yazhouliu / EUG

readme

EUG