Automation of CRS derived data series

[Duncan-Knox]

Requires further look-in to determine scale.

Contains the following:

donor_profile.recipient_bundle_2015 donor_profile.loans_grants_2015 donor_profile.sectors_over_time_2015 donor_profile.donor_bundle_2015

[k8hughes]

I am not sure if this needs to be a new issue or if it is correct to note here. @akmiller01 we need to find the dimension tables that were used in Jan or Feb. @Duncan-Knox can give more info.

[akmiller01]

Here's a full list of all the existing dimension tables in the DDW:

> dims = schemaListTables("dimension")
> dims[order(dims)]
 [1] "deflator_metadata"                                             
 [2] "di_id"                                                         
 [3] "di_id_to_iso_3166_1_map"                                       
 [4] "di_itep_channel"                                               
 [5] "di_itep_sector"                                                
 [6] "imf_weo_country"                                               
 [7] "imf_weo_country_to_di_id_map"                                  
 [8] "imf_weo_ncu_deflator_2014_2016_apr_pivoted"                    
 [9] "imf_weo_ncu_deflator_2014_2016_oct_pivoted"                    
[10] "imf_weo_ncu_deflator_2015_2017_apr_pivoted"                    
[11] "imf_weo_usd_deflator_2014_2016_apr_pivoted"                    
[12] "imf_weo_usd_deflator_2014_2016_oct_pivoted"                    
[13] "imf_weo_usd_deflator_2015_2017_apr_pivoted"                    
[14] "iso_3166_1"                                                    
[15] "master_oecd_deflator"                                          
[16] "master_oecd_deflator_2016_12_20"                               
[17] "master_oecd_deflator_pivoted"                                  
[18] "master_oecd_deflator_pivoted_2016_12_20"                       
[19] "oda_bundle"                                                    
[20] "oecd_crs_channel_code_5_digit"                                 
[21] "oecd_crs_channel_code_5_digit_to_di_itep_channel_map"          
[22] "oecd_crs_channel_code_5_digit_to_itep_channel_web_id_map"      
[23] "oecd_crs_donor"                                                
[24] "oecd_crs_purpose_code_5_digit"                                 
[25] "oecd_crs_purpose_code_5_digit_to_di_itep_sector_map"           
[26] "oecd_crs_purpose_code_5_digit_to_itep_sector_web_id_map"       
[27] "oecd_crs_recipient"                                            
[28] "oecd_crs_sector_code_3_digit"                                  
[29] "oecd_crs_sector_code_3_digit_to_di_itep_sector_map"            
[30] "oecd_crs_sector_code_3_digit_to_itep_sector_web_id_map"        
[31] "oecd_dac_1_donor"                                              
[32] "oecd_dac_2a_donor"                                             
[33] "oecd_dac_2a_recipient"                                         
[34] "oecd_dac_2b_donor"                                             
[35] "oecd_dac_2b_recipient"                                         
[36] "oecd_deflator_lookup"                                          
[37] "oecd_deflator_pivoted_dac_2016_01_20_non_dac_2016_01_20_dac_1" 
[38] "oecd_deflator_pivoted_dac_2016_01_20_non_dac_2016_01_20_dac_2a"
[39] "oecd_deflator_pivoted_dac_2016_04_20_non_dac_2016_04_20_dac_1" 
[40] "oecd_deflator_pivoted_dac_2016_04_20_non_dac_2016_04_20_dac_2a"
[41] "oecd_deflator_pivoted_dac_2016_07_20_non_dac_2016_07_20_dac_1" 
[42] "oecd_deflator_pivoted_dac_2016_07_20_non_dac_2016_07_20_dac_2a"
[43] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_crs"   
[44] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_dac_1" 
[45] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_dac_2a"
[46] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_dac_2b"
[47] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_11_21_dac_5" 
[48] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_crs"   
[49] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_1" 
[50] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_2a"
[51] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_2b"
[52] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_5" 
[53] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_12_crs"   
[54] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_1" 
[55] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_2a"
[56] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_2b"
[57] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_5" 
[58] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_crs"   
[59] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_1" 
[60] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_2a"
[61] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_2b"
[62] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_5" 
[63] "oecd_donor"                                                    
[64] "oecd_donor_to_di_id_map"                                       
[65] "oecd_donor_to_iso_3166_1_map"                                  
[66] "oecd_donor_type"                                               
[67] "oecd_loc_donor"                                                
[68] "oecd_loc_recipient"                                            
[69] "oecd_recipient"                                                
[70] "oecd_recipient_income_group"                                   
[71] "oecd_recipient_to_di_id_map"                                   
[72] "oecd_recipient_to_iso_3166_1_map"                              
[73] "oof_bundle"                                                    
[74] "time"                                                          
[75] "wb_wdi_country"                                                
[76] "wb_wdi_country_to_di_id_map"                                   
[77] "wb_wdi_country_to_imf_weo_country_map"                         
[78] "wb_wdi_usd_deflator_2013_2016_01_20_pivoted"                   
[79] "wb_wdi_usd_deflator_2014_2016_04_20_pivoted"                   
[80] "wb_wdi_usd_deflator_2014_2016_07_20_pivoted"                   
[81] "wb_wdi_usd_deflator_2014_2016_10_20_pivoted"                   
[82] "wb_wdi_usd_deflator_2014_2016_12_20_pivoted"                   
[83] "wb_wdi_usd_deflator_2015_2016_12_20_pivoted"                   
[84] "wb_wdi_usd_deflator_2015_2017_04_25_pivoted"

[Duncan-Knox]

Hi @akmiller01 thanks for sharing. The ones I was raising which had me confused around versions were the following:

Channels:

[4] "di_itep_channel" [20] "oecd_crs_channel_code_5_digit"
[21] "oecd_crs_channel_code_5_digit_to_di_itep_channel_map"
[22] "oecd_crs_channel_code_5_digit_to_itep_channel_web_id_map"

Sectors and purpose codes:

[5] "di_itep_sector"
[24] "oecd_crs_purpose_code_5_digit"
[25] "oecd_crs_purpose_code_5_digit_to_di_itep_sector_map"
[26] "oecd_crs_purpose_code_5_digit_to_itep_sector_web_id_map"
[28] "oecd_crs_sector_code_3_digit"
[29] "oecd_crs_sector_code_3_digit_to_di_itep_sector_map"
[30] "oecd_crs_sector_code_3_digit_to_itep_sector_web_id_map"

These had been updated more recently, however it looked like the ones coming through, especially for channels were older versions. Any help here in recovering or finding the newer version would be very helpful. Please let me know if this makes sense as a query.

[akmiller01]

@Duncan-Knox , attached is the current state of all of the dimensions listed above. If these dimension tables are up to date, but the code is still not creating the correct output, we'll need to adjust the code rather than the dimension tables. dims.xlsx

[Duncan-Knox]

@akmiller01 thank you. The dimension tables attached are not up to date.

[Duncan-Knox]

Complete for digital profiles. Amendments required for any PDF profile data. Put on hold given PDF prioritisation has been decreased in favour of digital profiles.