Add Nighttime light data to satellite features

[ChristinaLast]

Description

Air pollution exposure is known to correlate with Nighttime Light (a proxy for human activity captured through satellite images (Zeng 2021, Zeng 2023). We want to incorporate night time light into our method of feature generation to predict air pollution.

A developer can generate local air pollution predictions from satellite data configurations which capture the geographic relationship between nighttime light and air pollution, which improves the model accuracy.

Environment

The feature will be required to work for the following configurations:

• OpenAQ Engine Version: e.g., v.0.1.0
• Operating System: Ubuntu 20.04
• Additional Details:
  • You will need to register your own Google Service Account and a Service Account Private Key.
  • This project uses Google Earth Engine to generate satellite features. You will need to gain access to Earth Engine to build features.
  • After a model is successfully built, you can use your MLFlow instance to push the new model to a registry.

Screenshots or Log Files

Currently the structure of our satellite configuration is as follows:

1. After completing the registration of your google account, export your credentials

   export EE_API_KEY=your-ee-api-key
   export EARTHENGINE_CREDENTIALS=path/to/your/private_key.json
   export SERVICE_ACCOUNT_EMAIL=your-service-account@your-ee-project.iam.gserviceaccount.com

2. All features from satellites are built in openaq_engine/src/features/build_features.py here:

   def _add_ee_features(self, df: pd.DataFrame) -> pd.DataFrame:
       """
       Add Earth Engine (EE) features to the DataFrame.
   
       Parameters
       ----------
       df : pd.DataFrame
           The DataFrame to which EE features will be added.
   
       Returns
       -------
       pd.DataFrame
           The DataFrame with EE features added.
       """
       return EEFeatures.from_dataclass_config(EEConfig()).execute(
           df, save_images=False
       )

3. In openaq_engine/config/model_settings.py you will need to update EEConfig.ALL_SATELLITES. You will need to select the relevant bands.

   
   @property
   def ALL_SATELLITES(self) -> zip(List[str], List[str]):  # type: ignore
       """Return varying satellites to be fed into the model"""
       return zip(
           [
               self.AOD_IMAGE_COLLECTION,
               self.LANDSAT_IMAGE_COLLECTION,
               self.NIGHTTIME_LIGHT_IMAGE_COLLECTION,
               self.METEROLOGICAL_IMAGE_COLLECTION,
               self.POPULATION_IMAGE_COLLECTION,
               self.LAND_COVER_IMAGE_COLLECTION,
           ],
           [
               self.AOD_IMAGE_BAND,
               self.LANDSAT_IMAGE_BAND,
               self.NIGHTTIME_LIGHT_IMAGE_BAND,
               self.METEROLOGICAL_IMAGE_BAND,
               self.POPULATION_IMAGE_BAND,
               self.LAND_COVER_IMAGE_BAND,
           ],
           [
               self.AOD_IMAGE_PERIOD,
               self.LANDSAT_PERIOD,
               self.NIGHTTIME_LIGHT_PERIOD,
               self.METEROLOGICAL_IMAGE_PERIOD,
               self.POPULATION_PERIOD,
               self.LAND_COVER_PERIOD,
           ],
           [
               self.AOD_IMAGE_RES,
               self.LANDSAT_RES,
               self.NIGHTTIME_LIGHT_RES,
               self.METEROLOGICAL_IMAGE_RES,
               self.POPULATION_IMAGE_RES,
               self.LAND_COVER_IMAGE_RES,
           ],
       )

4. Adjust TrainerConfig.All_MODEL_FEATURES to include your selected bands

   All_MODEL_FEATURES = []

5. re-run the pipeline to build the new model with your additional features:

   openaq-engine run-pipeline /path/to/your/models/dir /path/to/your/plots/dir --pollutant pm25 --country {your-country-of-choice} --source [openaq-api/openaq-aws]

   Additional Context

Earth Engine provides many datasets with Nighttime Light, it is your job to find the most suitable for the task!