Closed hannahker closed 6 months ago
The action should fail when R errors out
This is an issue with using find
in GitHub Actions. This is why in the original file we were using globstar
. See this https://apple.stackexchange.com/questions/49042/how-do-i-make-find-fail-if-exec-fails
Connection errors when running action (for example)
Does this work when running locally?
un_geodata.geojson
: See #88
idmc_country_links.parquet
and indicator_mapping.parquet
. I put those together manually, just checking all country links on the IDMC website, and then piecing together indicator_mapping.parquet
as I was developing. I think we can hardcode an update script that we adjust as necessary. A really nice and easy trick is that you can generate the code to create the data frame in R super easily. Just load it in, save to df
and then use dput(df)
. However, it's not that clean / easy to update, whereas dplyr::tribble()
is a nice way to rowwise create data. I think we should do it for sure.
country_taxonomy_backup.parquet
Remove!
@caldwellst
Aha nice find!
This is an issue with using find in GitHub Actions. This is why in the original file we were using globstar. See this https://apple.stackexchange.com/questions/49042/how-do-i-make-find-fail-if-exec-fails
It does! I'm thinking that this could be from the {}
in the DSCI_AZ_ENDPOINT
environment variable. I'm thinking we can just hard-code this in the code itself since this doesn't contain any sensitive data.
Does this work when running locally?
@caldwellst this should be ready for final review now. As discussed, we'll come back to the ACLED data issues in future work. For now, I've left the bash command as using find
, so that this script can run through all other update files, despite the ACLED data failing. See #104
Minor updates to ensure that the static data update runs from the GitHub action with informative logging and status updates.
Issues still to resolve:
If we're assuming that all files under
inputs/
in the storage container should be updated by this workflow:adm0/
cities/
country_info.parquet
hrp_countries.parquet
iso3_map_settings.json
centroids/
-- Updates work locally but not in GH runnerindicator_mapping.parquet
idmc_country_links.parquet
acled_info.parquet
-- Updates work locally but not in GH runnerun_geodata.geojson
-- Updated manually and used in the code. Can this be updated programmatically?country_taxonomy_backup.parquet
-- Removing~region_bbox.parquet
-- Removing~region_bbox.geojson
-- Removing~