muthukumaranR
commented
2 years ago can we add the actual shapefile associated with this event in the visualizations ?
Closed ghltshubh closed 2 years ago
muthukumaranR
commented
2 years ago can we add the actual shapefile associated with this event in the visualizations ?
ghltshubh
commented
2 years ago Github doesn't support .tif files so changed to .png
smoke_wmts_ref_label
smokev4_6b_ref_extended_label
ghltshubh
commented
2 years ago The current threshold value is set at 0.5 therefore any pixels with probability higher than 0.5 are labeled as smoke and vice-versa.
We compare the outputs from the old model in production and the new model for different values of the probability threshold ranging from 0.1 - 0.9 with an increment of 0.1. Even at lower probability threshold of 0.1 as seen in the animation below, the old model tends to have higher number of false positives. It also has lower sensitivity and thus unable to detect thinner plumes.
ghltshubh
commented
2 years ago The new model has been trained on WMTS endpoint data and is able to detect thinner smoke plumes at lower threshold values as seen in the animation below. It also shows lower FP rate across all threshold values relative to the old model.
ghltshubh
commented
2 years ago 
Thinner plumes are captured in lower threshold (t = 0.1)
ghltshubh
commented
2 years ago Predicting on smoke event: 06/29/2021
============================================================================================
xhagrg
commented
2 years ago @kaulfusa , if this is reasonable, can we push this to phenomena portal? We could run some more days in the portal itself and check for any false positives...
Motivation
We had been using a model that was trained on the rasterized values obtained from raw NetCDF files for smoke detection. This caused some issues while running in the production setting. We had a lot of false positives. To handle this, we will need to retrain the model with the same source of data.
Background
Current smoke detection uses NN segmentation technique to detect smoke in rasterized images. To train a model we provide images along with their masks or labels that demarcate the smoke regions in those images using Intersection Over Union metric:
IoU = TP/(TP+FN+FP)For each pixel in an input image the model spits out a probability for that pixel being a smoke pixel or not. We choose a threshold value, say 0.5, as a cutoff and assign all the pixels with probability higher than this cutoff as smoke.
Training data source
False positive comparison technique
Currently, we are thresholding the output of models at probability value of 0.5 which means that any values equal to 0.5 and above are marked as smoke and below 0.5 are marked as no-smoke. This threshold probability is an important parameter that can be tuned for values between 0 and 1.
Therefore, we compare the 3 band outputs from both old and new model for threshold values between 0 - 1 with an increment of 0.1 for both WMTS and non-WMTS endpoint datasets.
The bandwise outputs are available: https://drive.google.com/drive/u/2/folders/1CGAth8SlykuymBb6cqkDhMC5IXPRqLMw
Analysis