Additional data layers and feature engineering

[emmamendelsohn]

• [x] Pull in immunity layer. This reflects outbreaks that happened prior to the current season.

• [ ] Separately, create a layer for whether there has been an outbreak in the region within the last year, as a first pass for capturing spread. (In a future iteration this could be something like distance to nearest outbreak in the last year.)

  When averaging to the ADM level:

• [ ] Sum taxa population
• [ ] Most likely remove slope and aspect - they become less relevant/interpretable at the ADM level
  • [ ] Average immunity? (Note, the immunity layer could potentially be regenerated at the ADM level and have the parameters tuned in the model)

[n8layman]

@emmamendelsohn is this similar to the outbreak history layer @noam started in that immunity is inferred from previous outbreak case data? Or is the immunity layer a separate seroprevalence dataset as suggested in #79? If based on case counts this will be included in PR #76 which contains daily outbreak histories in both the short and long term. In these histories, once an outbreak occurs the impact has an exponential decline, both spatially and as time progresses.

As an aside, SpatRasters produced with terra::writeRaster(..., gdal=c("COMPRESS=LZW")) were nearly half the size as when saving long-form data as parquet using compression = "gzip", compression_level = 5. Both .parquet and .tif versions are in the data/outbreak_history_dataset/ folder.

[emmamendelsohn]

Yes immunity layer is Noam's outbreak layer.

We primarily chose parquets for the ability to interact with the data outside of memory using arrow. Smaller rasters are good but before making any switch, make sure you can do all the data processing on them.

[n8layman]

I don't plan on switching, particularly for the full dataset tibble but it was interesting to observe and something to think about for future projects. Would tiffs avoid the parquet issue with AWS and targets?

[n8layman]

@kevinolival @rostal