Speckle2Void is a self-supervised Bayesian despeckling framework that enables direct training on real SAR images. This method bypasses the problem of training a CNN on synthetically-speckled optical images, thus avoiding any domain gap and enabling learning of features from real SAR images.
This repository contains python/tensorflow implementation of Speckle2Void, trained and tested on the TerraSAR-X dataset provided by ESA archive.
BibTex reference:
@ARTICLE{2020arXiv200702075B,
author = {{Bordone Molini}, Andrea and {Valsesia}, Diego and {Fracastoro}, Giulia and
{Magli}, Enrico},
title = "{Speckle2Void: Deep Self-Supervised SAR Despeckling with Blind-Spot Convolutional Neural Networks}",
journal = {arXiv e-prints},
keywords = {Electrical Engineering and Systems Science - Image and Video Processing, Computer Science - Computer Vision and Pattern Recognition},
year = 2020,
month = jul,
eid = {arXiv:2007.02075},
pages = {arXiv:2007.02075},
archivePrefix = {arXiv},
eprint = {2007.02075},
primaryClass = {eess.IV}
}
Make sure you have Python3 and all the required python packages installed:
pip install -r requirements.txt
Download the TerraSAR-X products from the ESA EO products online search service
Pre-process the dataset through the speckle decorrelator explained in the paper: Blind speckle decorrelation for SAR image despeckling. The blind-spot networks work properly if the noise is spatially decorrelated. The SAR imaging system correlates the speckle noise in SAR images during acquisition. A speckle decorrelator is needed before performing despeckling.
Place in the training directory a bunch of 10000x10000 decorrelated complex SAR images and one in the test directory. During traning some 1000x1000 patches, extracted from the 10000x10000 test image, will be used as testing images.
In Speckle2Void-training.ipynb the Speckle2V object is instantiated and some parameters are required:
"dir_train" : directory with training data.
"dir_test" : directory with test data.
"file_checkpoint" : checkpoint for loading a specific model. If None, the latest checkpoint is loaded.
"batch_size" : size of the mini-batch.
"patch_size" : size of the training patches
"model_name" : starting name of the directory where to save the checkpoints.
"lr" : learning rate.
"steps_per_epoch" : steps for each epoch
"k_penalty_tv" : coefficient to weigh the total variation term in the loss
"norm" : normalization
"clip" : intensity value to clip the SAR images
"shift_list" : list of the possible shifts to apply to the receptive fields at the end of the network. For example [3,1].
"prob" : list of the probabilities for choosing the possible shifts. For example [0.9,0.1], 0.9 will be the probability of using shift equal to 3 and 0.1 of using shift 1.
"L_noise" : parameter L of the noise distribution gamma(L,L) used to model the speckle
The SAR denoiser training starts by default from the latest checkpoint found in './checkpoint/model_name' or from a specified checkpoint.
The _s2vcheckpoint directory contains the model used to produce the results of the paper.
Download sample test images from here and place them in the test_examples directory. To test the trained model on the test examples and estimated the clean versions run Speckle2Void-prediction.ipynb.
Speckle2Void is based on work by the Image Processing and Learning group of Politecnico di Torino: Andrea Bordone Molini (andrea.bordone AT polito.it), Diego Valsesia (diego.valsesia AT polito.it), Giulia Fracastoro (giulia.fracastoro AT polito.it), Enrico Magli (enrico.magli AT polito.it).