search

nish-kishore / sirfunctions

Key functions used by the SIR team
Other
0 stars 0 forks source link

Fx to update a local rds with the new download form `get_all_polio_data()` #97

Open nish-kishore opened 2 months ago

nish-kishore commented 2 months ago

A wrapper around get_all_polio_data() which takes in a single parameter: local_dataset.

The function can be called update_polio_data and will download the small polio dataset and merge in each dataset into the local_dataset while also guaranteeing deduplication and any other data errors. It will then return this new merged dataset. Maybe we can also cache the metadata somewhere and have it check against that before it performs an update?

mcuadera commented 1 month ago

The purpose of this new function is to append new data to the local dataset with the same structure as what is returned by get_all_polio_data(). Need to figure out the best way to handle deduplication and caching the metadata