In order to make the assembly of the historical database transparent and reliable, it is necessary to reduce the cleaning scripts to only dataset-specific operations.
For instance, the conversion of country names to ISO 3166 codes should not be repeated in each cleaning script, but implemented as a general function that is applied to each input data set.
In order to make the assembly of the historical database transparent and reliable, it is necessary to reduce the cleaning scripts to only dataset-specific operations.
For instance, the conversion of country names to ISO 3166 codes should not be repeated in each cleaning script, but implemented as a general function that is applied to each input data set.