samapriya / awesome-gee-community-datasets

Community Datasets added by users and made available for use at large
https://gee-community-catalog.org/
Creative Commons Attribution 4.0 International
750 stars 119 forks source link

GAN-based Synthetic Monthly VIIRS-like NightTime Lights for India 1992-2012 #260

Closed spatialthoughts closed 2 months ago

spatialthoughts commented 2 months ago

Contact Details

ujaval@spatialthoughts.com

Dataset description

Mehak Jindal, Prasun Kumar Gupta, S.K. Srivastav, Generation of monthly VIIRS nighttime lights time-series (1992–2013) images using deep learning (cGAN) technique, Remote Sensing Applications: Society and Environment, Volume 35, 2024, 101263, ISSN 2352-9385, https://doi.org/10.1016/j.rsase.2024.101263.

Data Download from https://zenodo.org/records/7854534

Monthly nighttime lights (NTL) can clearly depict an area's prevailing intra-year socio-economic dynamics. The Earth Observation Group at Colorado School of Mines provides monthly NTL products from the Day Night Band (DNB) sensor on board the Visible and Infrared Imaging Suite (VIIRS) satellite (April 2012 onwards) and from Operational Linescan System (OLS) sensor onboard the Defense Meteorological Satellite Program (DMSP) satellites (April 1992 onwards). In the current study, an attempt has been made to generate synthetic monthly VIIRS-like products of 1992-2012, using a deep learning-based image translation network. Initially, the defects of the 216 monthly DMSP images (1992-2013) were corrected to remove geometric errors, background noise, and radiometric errors. Correction on monthly VIIRS imagery to remove background noise and ephemeral lights was done using low and high thresholds. Improved DMSP and corrected VIIRS images from April 2012 - December 2013 are used in a conditional generative adversarial network (cGAN) along with Land Use Land Cover, as auxiliary input, to generate VIIRS-like imagery from 1992-2012. The modelled imagery was aggregated annually and showed an R2 of 0.94 with the results of other annual-scale VIIRS-like imagery products of India, R2 of 0.85 w.r.t GDP and R2 of 0.69 w.r.t population. Regression analysis of the generated VIIRS-like products with the actual VIIRS images for the years 2012 and 2013 over India indicated a good approximation with an R2 of 0.64 and 0.67 respectively, while the spatial density relation depicted an under-estimation of the brightness values by the model at extremely high radiance values with an R2 of 0.56 and 0.53 respectively. Qualitative analysis for also performed on both national and state scales. Visual analysis over 1992-2013 confirms a gradual increase in the brightness of the lights indicating that the cGAN model images closely represent the actual pattern followed by the nighttime lights. Finally, a synthetically generated monthly VIIRS-like product is delivered to the research community which will be useful for studying the changes in socio-economic dynamics over time.

Earth Engine Snippet if dataset already in GEE

n/a

Enter license information

CC-BY-4.0

Keywords

viirs, dmsp, ntl, gan

Code of Conduct

samapriya commented 2 months ago

This has been completed but with issues about the file name and distribution. I have reached out to the author for clarification feel free to include any clarifications you might be aware of. For now there are more than 12 months per year and less than 12 months in a few years. Also some combinations of year and month repeat across different Model F though am not sure what that means. Here is sample code.

Feel free to reopen this ticket if you get any clarification on this

prasunkgupta commented 2 months ago

The original DMSP monthly product (from which the current product is derived) have been captured by a series of satellites (F10-F18 ), and have been made available by EOG between 1992 and 2014. The different colours in the image below show its availability over Indian landmass; image source). The green ticks show the 216 monthly images that were used in our paper, for the generation of the improved VIIRS-like product. The red crosses display the files that are available on EOG, however, due to clouds over the Indian region, the products had large spatial gaps, and could not be used for creation of this improved product. The blue cells indicate months for which data was absent on the EOG portal, at the time of creation of this improved product.

image

Hence there are can be multiple cases where:

  1. 12 months data is not available for an year, because either (a) the data was not available (from original source), e.g., F12_1994; or (b) the data was not available over India (from original source), e.g., F12_1995.
  2. More than 12 months data is available for an year, because (a) the data was captured by multiple satellites for 1 or more months (e.g., 1999 from F12 and F14).
  3. Exactly 12 months data is available for an year, but it is not for all months, because it is a combination of months from different satellites for the given year (e.g., 1994 from F10 and F12).

The data is not temporally continuous. Users are advised to use the monthly data appropriately.

samapriya commented 2 months ago

Thank you for those details here is the unlisted page for now for review https://gee-community-catalog.org/projects/syn_ntl/

Let me know if there are any changes. I will add the Sample code link before publication

prasunkgupta commented 2 months ago

Thank you for those details here is the unlisted page for now for review https://gee-community-catalog.org/projects/syn_ntl/

This looks awesome!

spatialthoughts commented 2 months ago

Thanks @samapriya for putting this together. Amazing work as always.