Benchmarks for AutoAlbument - AutoML for Image Augmentation.
Results
CIFAR-10 (Classification)
Augmentation strategy |
Top-1 Accuracy |
Top-5 Accuracy |
Baseline |
91.79 |
99.63 |
AutoAlbument |
96.02 |
99.91 |
SVHN (Classification)
Augmentation strategy |
Top-1 Accuracy |
Top-5 Accuracy |
Baseline |
98.31 |
99.68 |
AutoAlbument |
98.48 |
99.72 |
ImageNet (Classification)
- Model: ResNet-50.
- Baseline augmentation strategy:
- Resize an image to 256x256 pixels.
- Crop a random 224x224 pixels patch.
- Apply Horizontal Flip with probability 0.5.
- AutoAlbument augmentation strategy:
- Resize an image to 256x256 pixels.
- Crop a random 224x224 pixels patch.
- Apply AutoAlbument augmentation policies.
- Configs: AutoAlbument augmentation search | Baseline training | AutoAlbument training.
Augmentation strategy |
Top-1 Accuracy |
Top-5 Accuracy |
Baseline |
73.27 |
91.64 |
AutoAlbument |
75.17 |
92.57 |
Pascal VOC (Semantic segmentation)
- Model: DeepLab-v3-plus.
- Baseline augmentation strategy:
- Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
- If required, pad an image to the size 256x256 pixels.
- Apply Horizontal Flip with probability 0.5.
- AutoAlbument augmentation strategy:
- Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
- If required, pad an image to the size 256x256 pixels.
- Apply AutoAlbument augmentation policies.
- Configs: AutoAlbument augmentation search | Baseline training | AutoAlbument training.
Augmentation strategy |
mIOU |
Baseline |
73.34 |
AutoAlbument |
75.55 |
Cityscapes
- Model: DeepLab-v3-plus.
- Baseline augmentation strategy:
- Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
- If required, pad an image to the size 256x256 pixels.
- Apply Horizontal Flip with probability 0.5.
- AutoAlbument augmentation strategy:
- Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
- If required, pad an image to the size 256x256 pixels.
- Apply AutoAlbument augmentation policies.
- Configs: AutoAlbument augmentation search | Baseline training | AutoAlbument training.
Augmentation strategy |
mIOU |
Baseline |
79.47 |
AutoAlbument |
79.92 |
How to run the benchmarks
- Download datasets and put them in the following directory structure:
- Clone this repository.
- Run the
run.sh
script that will build a Docker image and train models using the following command:
./run.sh </path/to/data/directory> </path/to/outputs/directory>
e.g.
./run.sh ~/data ~/outputs
where
</path/to/data/directory>
is a path to a directory that contains datasets (e.g., a directory that contains folders imagenet
, pascal_voc
, etc)
</path/to/outputs/directory>
is a path to a directory that should contain outputs from a training pipeline, such as a CSV log with metrics and a checkpoint with the best model.