transportenergy / database

Tools for accessing and maintaining the iTEM model & historical databases
https://transportenergy.rtfd.io
GNU General Public License v3.0
23 stars 8 forks source link

T000: Missing data for the 22 countries #60

Open hlinero opened 3 years ago

hlinero commented 3 years ago

Data source: https://stats.oecd.org/index.aspx?queryid=79863

Total inland passenger transport = Road passenger transport by passenger cars + Road passenger transport by buses and coaches

There are 22 countries that have missing data for either the variables "Road passenger transport by passenger cars" or "Road passenger transport by buses and coaches" for certain years. Therefore, the total sum Road Passenger Transport is incorrectly reported. Below are the countries and years in which one of those variables is missing:

'Albania': 1970 to 1991 'Armenia': 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017 'Azerbaijan': 1970 to 1984 'Belarus': 1970 to 1993 'Bosnia-Herzegovina': 1970 to 1997 and 2007 to 2017 'Bulgaria': 1970, 1995 to 2018 'Canada': 2008 to 2009 'China': 1978, 1980, 1985, 1990 to 2017 'Croatia': 1970 to 2018 'Estonia': 1970 to 2018 'Georgia': 1976 to 2018 'India': 1990 to 2017 'Latvia': 1970 to 2018 'Mexico': 1970 to 2018 'Moldova': 1970 to 2018 'Montenegro, Republic of': 2001 to 2016 'Romania': 1970 to 2018 'Russian Federation': 1970 to 1990 'Switzerland': 1986 to 1997 'Turkey': 1970 to 2018 'Ukraine': 1990 to 2019 'United States': 1970, 1975

Currently, the sum Total inland passenger transport from these countries are removed in the code https://github.com/transportenergy/database/blob/master/item/historical/scripts/T000.py

The procedures to fix this problem are the following:

  1. ITF correct these errors in their data https://stats.oecd.org/index.aspx?queryid=79863
  2. Implement code change https://github.com/transportenergy/database/blob/master/item/historical/scripts/T000.py removing the cleaning scripts associated with this problem
  3. Regenerate a new merged dataset
RachelePoggi commented 3 years ago

I corrected the data in the database according to your suggestions. I did not delete data for these countries:

I need to check data for Bulgaria and Russian Federation because road passenger transport by car seems small.