Currently, as part of our data update workflow, we need to manually archive a grapher dataset after we have created a new version for it.
But (in principle) this could be done automatically after chart-sync is completed: If we have moved all indicators from one grapher dataset to its corresponding new version (and we have also checked that the indicators of the old grapher dataset are not used in any indicator-based explorer) we could also automatically set isArchived=1 on the old grapher dataset.
NOTE: After discussion, we thought it could be better to do it as part of the archiving CLI tool in ETL.
@pabloarosado It could have a flag --grapher to also archive the corresponding grapher datasets.
@Marigold Once we know this works well, and takes care of all dependencies and possible issues, it could be used by chart-sync process.
Technical notes
This issue is mainly about archiving grapher datasets after an update (i.e. setting isArchive=1). We'd need to check that this dataset does not have charts (and, if possible, that it's not used by indicator-based explorers).
We could consider if it should also archive etl steps automatically (we should have a working CLI). However, it's not clear what to do if it fails.
Currently, as part of our data update workflow, we need to manually archive a grapher dataset after we have created a new version for it.
But (in principle) this could be done automatically after chart-sync is completed: If we have moved all indicators from one grapher dataset to its corresponding new version (and we have also checked that the indicators of the old grapher dataset are not used in any indicator-based explorer) we could also automatically set
isArchived=1on the old grapher dataset.NOTE: After discussion, we thought it could be better to do it as part of the archiving CLI tool in ETL.
--grapherto also archive the corresponding grapher datasets.Technical notes
isArchive=1). We'd need to check that this dataset does not have charts (and, if possible, that it's not used by indicator-based explorers).