search

SWS-Methodology / faoswsTrade

World trade data processing for the FAO Statistical Working System
http://www.fao.org/economic/ess/ess-home/en/
5 stars 2 forks source link

Some country-year combinations are not valid #128

Open chrMongeau opened 7 years ago

chrMongeau commented 7 years ago

The module is returning some country-year combinations that should not exist (HT @sebastian-c), e.g., there is data for "Montenegro" and for "Serbia" in 2003 ("Serbia and Montenegro" for year <= 2005), "NethAntilles" in 2012 (when there should be separate data for "Curaçao", "Sint Maarten (Dutch Part)" and "Bonaire, Sint Eustatius and Saba" for year >= 2011), and "Sudan" in 2008 (that should be "Sudan (former)" for year <= 2011).

carola-f commented 7 years ago

You could use a reference table that has been developed for dissemination purposes that gives for each country code its StartYear and Endyear. It is available in FAOSTAT metadata or in T:\Team_working_folder\A\Classifications\CountryList\FAOSTAT_data_5-2-2017.csv

sebastian-c commented 7 years ago

The table itself also contains this and you should make sure that it is the same as the FAOSTAT one: GetCodeList("trade", "total_trade_cpc_m49", "geographicAreaM49")

     code              description selectionOnly      type  startDate endDate
  1:  835                 Zanzibar         FALSE   country         NA      NA
  2:  728              South Sudan         FALSE   country 2011-07-11      NA
  3:  248            Åland Islands         FALSE   country         NA      NA
  4:  239 SouthGeorgia/Sandwich Is         FALSE   country         NA      NA
  5:  535 Bonaire/S.Eustatius/Saba         FALSE   country         NA      NA
chrMongeau commented 7 years ago

I used:

GetCodeList(domain = 'faostat_one', dataset = 'FS1_SUA_UPD', dimension = 'geographicAreaFS')

given that reporters' codes were FAO.

The only difference relevant with respect to the file in T:\Team_working_folder\A\Classifications\CountryList\FAOSTAT_data_5-2-2017.csv that I see has to do with Netherlands Antilles: in the FS1_SUA_UPD dataset its endYear is 2099, while it is 2010 in the CSV file.

chrMongeau commented 7 years ago

I just asked news about this, so that it can be updated (the endYear for Netherlands Antilles should actually be 2010: https://en.wikipedia.org/wiki/Netherlands_Antilles).

chrMongeau commented 6 years ago

It was confirmed that in FAOSTAT domain Netherland Antilles is (correctly) not reported after 2010. Other domains do.