Oil detection with neural networks
This work has been realized as part of my internship at the IMT / CEFREM. The state of the project at the end of it is available on the endInternship branch. To further improve it I have decided to continue to develop new code on the dev branch
Summary
2.2 Image patches classification
1. Data
1.1 Network input data
The rasterio will allow the user to open raster files which consists of 3 files per image : .hdr, .img and .bsd :
- one file
.img
It is the image written in a binary format.
- one file
.hdr
It is the header containing the metadata of the image : (documentation here)
Raw images .tiff has been opened with snap (VV (vertical polarisation) channel) preprocessed by the snap software with several modules:
pipeline used at the beginning of the internship (cf code of the folder to see effectively which preprocessing steps are used):
We have(will for the moment) reproduced it as a python pipeline in order to change the preprocessing, remove one step to see the influence on the final precision of the AI.
1.2 Labels
There are 4 000 images manually segmented
To highlight which regions of the image belong to which category, CEFREM members have drawn polygons on the image with qgis and indicated at which category pixels inside them belong. We have access to their vertices with pixel coordinates or gps coordinates with the .shp file. We can find the caategory of each polygon in the .dbf file.
These files can be opened in python with the geopandas package
So, it is necessary to determine with python to which category belong which pixels thanks to the polygons.
In the future we may use another shapefile to distinguish the land from the sea. It might be possible to annotate the boats by using data from the marinetraffic website. (But there are API quotas....) or to use AIS data.
In the future, it will be possible to calculate the wind field of the raster image thanks to snap (cf Radar > SAR Applications > Ocean Applications > Wind Field Estimation) It will be possible to apply a speckle operation before in order to smooth the ouput or a moving average. We can also later filter small objects...
We will be able to add land labels thanks to premade maps. We just need to adjust them (put a 2000-5000 meters margin). It has been done directly in QGIS.
1.3 Notes
- The network may have difficulties to distinguish:
- the 2 types of oil discharge (spill or seep)
- plankton can create strings that look like oil discharge. However, they can be differentiated by their edges: the transition of luminosity on the image between the sea and an oil discharge will be sharper than with the plankton. That is why diminishing the resolution of the image can be a problem
- We will eventually add more categories as plankton by thresholding the image.
- The confidence level for each polygon is also available in the
.dbsunder theìndicecolumn. It can be interesting to integrate it in future models. - The "raw" (before any preprocessing of this project) images are 25 000 px width. It might become a bottleneck for the training (to be tested)
- Potential solutions :
- Cut the image into patches (500 px widthpar for instance, not less): ⚠️ be cautious not to cut too many oil discharge features
- Reduce the resolution: ⚠️ potential problems with the plankton
- Potential solutions :
2 Objectives
2.1 General objectives
- Create a segmentation map of the image
There are 3 categories (classes) possible (at the initial state of the project)
-
Oil discharge type 1: oil spill: caused by boats
-
Oil discharge type 2 : seep: natural discharge
-
The rest
-
Be able to tell the coordinates of each input image and their resolution (for eventual debugging purposes (plankton))
-
TODO later: Keep the mean of the sea part to the same level for each image
2.2 Image patches classification
We take the image and split it into smaller regions (named patches). The network then has to tell which class is present on this image. As several classes may be on the same patch, we can predict the probability that each class is on the image. Thus, we will have the following output vector :