search

iarai / Landslide4Sense-2022

Data description and baseline code for LandSlide4Sense 2022 competition
MIT License
83 stars 22 forks source link
competition dataset deep-learning landslide natural-hazard pytorch remote-sensing

readme

Landslide4Sense: Multi-sensor landslide detection competition & benchmark dataset

Contents

Landslide4Sense 2022

Logo

The Landslide4Sense competition, organized by Institute of Advanced Research in Artificial Intelligence (IARAI), aims to promote research in large-scale landslide detection from multi-source satellite remote sensing imagery. Landslide4Sense dataset has been derived from diverse landslide-affected areas around the world from 2015 through 2021. This benchmark dataset provides an important resource for remote sensing, computer vision, and machine learning communities to support studies on image classification and landslide detection. Interested in innovative algorithms for landslide detection using satellite imagery? Join us to help shape Landslide4Sense 2022!

Globally Distributed Landslide Detection

Landslides are a natural phenomenon with devastating consequences, frequent in many parts of the world. Thousands of small and medium-sized ground movements follow earthquakes or heavy rain falls. Landslides have become more damaging in recent years due to climate change, population growth, and unplanned urbanization in unstable mountain areas. Early landslide detection is critical for quick response and management of the consequences. Accurate detection provides information on the landslide exact location and extent, which is necessary for landslide susceptibility modeling and risk assessment. Recent advances in machine learning and computer vision combined with a growing availability of satellite imagery and computational resources have facilitated rapid progress in landslide detection. Landslide4Sense aims to promote research in this direction and challenges participants to detect landslides around the globe using multi-sensor satellite images. The images are collected from diverse geographical regions offering an important resource for remote sensing, computer vision, and machine learning communities.

Challenge Description

The aim of the competition is to promote innovative algorithms for automatic landslide detection using remote sensing images around the globe, and to provide objective and fair comparisons among different methods. The competition ranking is based on a quantitative accuracy metric (F1 score) computed with respect to undisclosed test samples. Participants will be given a limited time to submit their landslide detection results after the competition starts. The winners will be selected from the top three solutions in the competition ranking.

Special prizes will be awarded to creative and innovative solutions selected by the competition's scientific committee based on originality, generality, and scalability.

The competition will consist of two phases:

Phase 1 (April 1 - June 14): Participants are provided with training data (with labels) and additional validation images (without labels) to train and validate their methods. Participants can submit their landslide detection results for the validation set to the competition website to get feedback on the performance (Precision, Recall, and F1 score). The ranking of the submission will be displayed on the online leaderboard. In addition, participants should submit a short description of the methodology (1-2 pages) here using the IJCAI LaTeX styles, and Word templates.

Phase 2 (June 15 - June 20): Participants receive the test data set (without labels) and must submit their landslide detection results within 5 days from the release of the test data set. The submissions during that week will be limited to 10 times and only the F1 score will be displayed on the online leaderboard.

The winners of Phase 2 of the competition will be asked to write a 4-page IJCAI-style formatted manuscript that will be included in the CDCEO workshop. Each manuscript should describe the addressed problem, the proposed method, and the results. The winners will need to prepare a short pre-recorded video presentation for the workshop. The winners should also be present for a live Question-and-Answer period with the audience during the workshop.

The winners must submit the working code, the learned parameters, and the manuscript, and present their work in the CDCEO workshop at IJCAI-ECAI 2022 to receive the prizes in accordance with the terms and conditions of the competition.

Data Description

The Landslide4Sense dataset has three splits, training/validation/test, consisting of 3799, 245, and 800 image patches, respectively. Each image patch is a composite of 14 bands that include:

All bands in the competition dataset are resized to the resolution of ~10m per pixel. The image patches have the size of 128 x 128 pixels and are labeled pixel-wise.

Download links: training and validation.

Logo

The Landslide4Sense dataset is structured as follows:

├── TrainData/
│   ├── img/     
|   |   ├── image_1.h5
|   |   ├── ...
|   |   ├── image_3799.h5
│   ├── mask/
|   |   ├── mask_1.h5
|   |   ├── ...
|   |   ├── mask_3799.h5
├── ValidData/
|   ├── img/     
|   |   ├── image_1.h5
|   |   ├── ...
|   |   ├── image_245.h5
├── TestData/
    ├── img/     
        ├── image_1.h5
        ├── ...
        ├── image_800.h5

Note that the label files (mask files) are only accessible in the training set.

Mapping classes used in the competition:

Class Number Class Name Class Code in the Label
1 Non-landslide 0
2 Landslide 1

Baseline Code

This repository provides a simple baseline for the Landslide4Sense competition based on the state-of-the-art DL model for semantic segmentation, implemented in PyTorch. It contains a customizable training script for U-Net along with the dataloader for reading the training and test samples (see landslide_dataset.py in the dataset folder).

The provided code can be used to predict baseline results for the competition or as a comparison method for your solutions. Feel free to fork this repository for further use in your work!

Required packages and libraries:

To train the baseline model:

python Train.py --data_dir <THE-ROOT-PATH-OF-THE-DATA> \
                --gpu_id 0

Please replace <THE-ROOT-PATH-OF-THE-DATA> with the local path where you store the Landslide4Sense data.

The trained model will then be saved in ./exp/

To generate prediction maps on the validation set with the trained model:

python Predict.py --data_dir <THE-ROOT-PATH-OF-THE-DATA> \
               --gpu_id 0 \
               --test_list ./dataset/valid.txt \
               --snapshot_dir ./validation_map/ \
               --restore_from ./exp/<THE-SAVED-MODEL-NAME>.pth

Please replace <THE-SAVED-MODEL-NAME> with the name of your trained model.

Alternatively, our pretrained model is available at here.

The generated prediction maps (in h5 format) will then be saved in ./validation_map/

To generate prediction maps on the test set with the trained model:

python Predict.py --data_dir <THE-ROOT-PATH-OF-THE-DATA> \
               --gpu_id 0 \
               --test_list ./dataset/test.txt \
               --snapshot_dir ./test_map/ \
               --restore_from ./exp/<THE-SAVED-MODEL-NAME>.pth

The generated prediction maps (in h5 format) will then be saved in ./test_map/

Evaluation Metric

The F1 score of the landslide category is adopted as the evaluation metric for the leaderboard:

With the provided baseline method and the pretrained model, you can achieve the following result on the validation set:

Validation Set Precision Recall F1 Score
U-Net Baseline 51.75 65.50 57.82

Note that the evaluation ranking is ONLY based on the F1 score of the landslide category in both validation and test phases.

Submission Guide

For both validation and test phases, participants should submit a ZIP file containing the prediction files for all test images. Each pixel in the prediction file corresponds to the class category with 1 for landslide regions and 0 for non-landslide regions (similar to the reference data of the training set).

Specifically, the predictions for each test image should be encoded as a h5 file with the Byte (uint8) data type, and match the dimensions of the test images (i.e., 128×128).

The submitted ZIP file in the validation phase should be structured as follows:

├── submission_name.zip     
    ├── mask_1.h5
    ├── mask_2.h5
    ├── ...
    ├── mask_245.h5

The submitted ZIP file in the test phase should be structured as follows:

├── submission_name.zip     
    ├── mask_1.h5
    ├── mask_2.h5
    ├── ...
    ├── mask_800.h5

Sample command for the ZIP file generation:

cd ./validation_map
zip <THE-SUBMISSION-NAME>.zip ./*

Awards and Prizes

The winners of the competition will be selected from the top three ranked solutions and will be awarded the following prizes:

Timeline

Frequently Asked Questions

Citation

Please cite the following paper if you use the data or the codes: 

@article{L4S-2022,
  author={Ghorbanzadeh, Omid and Xu, Yonghao and Zhao, Hengwei and Wang, Junjue and Zhong, Yanfei and Zhao, Dong and Zang, Qi and Wang, Shuang and Zhang, Fahong and Shi, Yilei and Zhu, Xiao Xiang and Bai, Lin and Li, Weile and Peng, Weihang and Ghamisi, Pedram},
  journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
  title={The Outcome of the 2022 Landslide4Sense Competition: Advanced Landslide Detection From Multisource Satellite Imagery},
  year={2022},
  volume={15},
  number={},
  pages={9927-9942},
  doi={10.1109/JSTARS.2022.3220845}}