Covariates: Literature review

[MAKassien]

1. Search literature on similar models to define most commonly used covariates

[MAKassien]

Some papers to check out:

Two from the same group: An ensemble learning approach for estimating high spatiotemporal resolution of ground-level ozone in the contiguous United States https://doi.org/10.1021%2Facs.est.0c01791 An ensemble-based model of PM2.5 concentration across the contiguous United States with high spatiotemporal resolution https://doi.org/10.1016/j.envint.2019.104909

Predicting Fine Particulate Matter (PM2.5) in the Greater London Area: An Ensemble Approach using Machine Learning Methods https://doi.org/10.3390/rs12060914

Deep Ensemble Machine Learning Framework for the Estimation of PM2.5 Concentrations https://doi.org/10.1289/EHP9752

[mitchellmanware]

An ensemble learning approach for estimating high spatiotemporal resolution of ground-level ozone in the contiguous United States

https://pubs.acs.org/doi/10.1021/acs.est.0c01791 Link for Supporting Information with covariate details: https://pubs.acs.org/doi/suppl/10.1021/acs.est.0c01791/suppl_file/es0c01791_si_001.pdf

Model

• Pollutant: Ozone
• Spatial extent: contiguous United States
• Spatial resolution: 1km²
• Temporal resolution: daily
• Temporal extent: 2000 - 2016

Covariates

• 169 total covariates
• Individual variates were included multiple times at varying spatial resolution (30m, 100m, 1km, 10km) as some covariates outside of the immediate 1km grid cell may be relevant

  For example, NO₂ and VOC emissions 10km may be relevant, and land use and meteorology predictors are surrogates for such things. Finally, we used a finer scale because some predictors are related to NO emissions which quench O₃, and these vary on a fine spatial scale.

• Meteorological variables
  • Obtained from NCEP Reanalysis 1
  • Dataset contains most of (if not all) meteorological variables that are needed, but has a low spatial resolution (32km²)
  • Using NCEP Reanalysis 1 may have quick pre-processing time due to common source and lower resolution but is limited by low resolution
  • Data points may be available via other sources at higher spatial resolution but will take longer to process
• Land cover variables
  • Model uses two different sources to measure land use: National Land Cover Database (NLCD) and MODIS MCD12Q1
  • Also includes MODIS NDVI as a "complement for vegetation areas"
  • Any potential benefits to having two measures of land use? Should we use one or both/more than one land cover measure?
• Precursors
  • Model includes SO₂, NO₂, NO_x, VOC and CO data from various sources to measure O₃ precursors to improve model
  • Inclusion of precursors may depend on our pollutant. I am not familiar with which needs to be included for each pollutant.

[mitchellmanware]

An ensemble-based model of PM_2.5 concentration across the contiguous United States with high spatiotemporal resolution

https://www.sciencedirect.com/science/article/pii/S0160412019300650?via%3Dihub Link for Supporting Information: https://ars.els-cdn.com/content/image/1-s2.0-S0160412019300650-mmc1.docx

Model

• Pollutant: PM2.5
• Spatial extent: contiguous United States
• Spatial resolution: 1km²
  • Downscaled to 100m² grid cells using land-use variables
• Temporal resolution: daily
• Temporal extent: 2000 - 2015

Covariates

• Meteorological variables
  • Also obtained from NCEP Reanalysis 1
  • Fewer meteorological variables included
  • air temperature; accumulated total precipitation, downward radiation flex, accumulated total evaporation, planetary boundary layer height, low cloud area fraction, precipitable water for the entire atmosphere, pressure, specific humidity at 2 m, visibility, wind speed (computed as vector sum of u-wind at 10 m and v-wind at 10 m), medium cloud area fraction, high cloud area fraction and albedo
• Chemical transport model
  • GEOS-Chem daily predictions of total PM_2.5 mass and mass concentrations of PM_2.5 components
  • 0.5° x 0.625° resolution
  • CMAQ model
  • EPA
  • 12km² resolution over the United States
• Land use variables
  • National Land Cover Database (NLCD) data
  • Area of each land use type calculated for 1 km² grid cells
  • Road density
  • Data from US Census Bureau
  • Road density per 1km² grid cell calculated
  • NDVI from MODIS

Notes

• Both used NCEP Reanalysis 1 meteorological data
• Model only included NLCD land cover data
  • Previous article used NLCD and MODIS-derived land use data
• NLCD land-use type, elevation, road density and restaurant density are standard land-use variables

[mitchellmanware]

Predicting fine particulate matter (PM_2.5 in the greater London area: an ensemble approach using machine learning methods

https://www.mdpi.com/2072-4292/12/6/914

Model

• Pollutant: PM_2.5
• Spatial extent: greater London area
• Spatial resolution 1km²
• Temporal resolution: daily
• Temporal extent: 2005 - 2013

Notes

• Included variables that were not considered in previous studies
  • population density, distance from nearest airport, distance to nearest major road, distance to nearest bus stop, number of bus stops, light at night, number of buildings and average building height
  • as this study was conducted in the UK, dataset availability may partially explain difference in variables
  • most meteorological, land use and atmospheric covariates are the same

[mitchellmanware]

Deep ensemble machine learning framework for the estimation of PM_2.5 concentrations

https://ehp.niehs.nih.gov/doi/10.1289/EHP9752 Link to Supplemental Materials with covariate details: https://ehp.niehs.nih.gov/action/downloadSupplement?doi=10.1289%2FEHP9752&file=ehp9752.s001.acco.pdf

Model

• Pollutant: PM_2.5
• Spatial extent: Italy
• Spatial resolution: 1kim²
• Temporal resolution: daily
• Temporal extent: 2015 - 2019

Notes

• Included population density measure (population per 1 km₂
• Many of the same meteorological, land use and time variables
• Uses one measure of aerosol optical depth (blue band at 470nm at 1km²) from MODIS' Terra + Aqua combined product
  • Previous studies have used both 470nm and 550nm from Aqua and Terra separately
  • Latest MODIS AOD product has combined Terra + Aqua product available (

[eva0marques]

Model validation methods and scores

Requia et al. 2020: (O2 predictions)

- Methods

• 10-fold CV for each ML model (monitoring sites divided in 10 splits, train with 9, test on the last one)
• compute R2, spatial R2 (R2 on annual mean diff), temporal R2 (R2 on the diff of delta to annual mean at the site)
• CV on different samples: whole period, per year, region, season, population density
• model uncertainty: monthly standard deviation of residuals

- Results

• higher uncertainty in the Eastern part of USA, especially in summertime, low uncertainty in wintertime. No significant performance diff between high populated areas and countryside.
• ensemble model is not significantly better than separate RF, XGBoost and NN (gradient boosting is slightly better than the others but NN is better at predicting high concentrations)
• R2>0.88 for each year, RMSE<5.7ppb (decreases significantly with the years)
• harder to predict higher values (>120ppb)

Di et al. 2019

(methodologies very close to Requia et al., Di was second author in the previous paper)

- Methods

• Approximately the same than in previous paper.
• sd predicted based on covariates. Methodology is not very clear.
• provide a table with the relative contribution of predictor variables for each ML method

- Results

• cross validated R2 by year: from 0.75 to 0.9, average of 0.86
• spatial R2 from 0.73 to 0.91, average of 0.89
• average RMSE: 1.26 microgram/m3 spatially and 2.53microgram/m3 temporally
• RF and NN better than GB, ensemble model outperforms very slightly
• harder to predict the Eastern part of the US

Interesting: to leverage spatial autocorrelation, they included spatially lagged monitored PM2.5 as predictor variable.

Yazdi et al. (2020)

- Methods

• 10-fold CV
• RMSE and R2, spatial and temporal R2, intercept and slope of pred vs obs
• Moran's I (spatial autocorrelation) on the residuals
• variable importance based on SHAP method (they include "average city-wide PM2.5" as a predictor and it is the more influential predictor)

- Results

• RF performs better than GB, KNN (overall and temporally) but GB better for spatial predictability.
• R2 for the ensemble model: between 0.74 (summer) and 0.85 (winter)
• RMSE for the ensemble model: 4.23 microgram/m3
• spatial residuals: no discernable pattern
• Moran's I suggest that the dispersion of residuals is random (no spatial trend)

Yu et al. (2022)

- Methods

• hold-out method and 10-fold CV (repeated 10x for each fold)
• dissimilarity index DI (monitoring sites spatial distribution very imbalanced) : normalized and weighted minimum distance to the nearest training data point divided by the average distance within the training data
• spatial CV and temporal CV to investigate spatial and temporal overfitting (per region and with separate days to see forecasting ability)
• comparison of deep ensemble ML model (DEML) to other benchmark basic models

- Results

• DEML R2: 0.85 and RMSE: 5.38 microgram/m3 (better than benchmark models)
• worse results in summer

To sum up:

• they all use 10-fold CV
• best results obtained with Requia et al. (2020) and Di et al. (2019), the most influential predictor was spatially lagged monitored PM2.5 (kind of nearest neighbors value)
• necessity to create scoring methods to evaluate spatial and temporal consistency. Yazdi et al. (2020) used Moran's index on the residuals to check if there was a spatial pattern
• interesting: higher values are of the utmost importance for human health. Shouldn't we create a specific metric and a loss function to optimize the prediction of these values? Shouldn't we create another model to predict the probability to go above a threshold?