alexandersimoes / oec

The website powering the Observatory of Economic Complexity
atlas.media.mit.edu
159 stars 52 forks source link

Data discrepancies in SITC classification #44

Open mkvp opened 10 years ago

mkvp commented 10 years ago

Hello. First of all, I would like to express my deep appreciation of your work. Working with your data and with your interactive visualization, there is something I haven’t been able to understand. It’s about the number on export and imports when SITC classification is applied. I’ve compared these numbers with the ones of UN-COMTRADE database, where I supposed they came from, and I found huge differences. For instance, the assessment of total exports from USA in 2012 in STIC classification is $2.59T, while it’s $1.23T in HS classification. Furthermore if you search in COMTRADE database, you can find different numbers for the same SITC code. Here you have an example.

Can anyone, please, help me to understand such a big difference? Where I’m wrong? Thank you.

*COMTRADE search and results: http://comtrade.un.org/api/get?max=50000&type=C&freq=A&px=S4&ps=2012&r=842&p=0&rg=2&cc=AG4

{"pfCode":"S4","yr":2012,"period":2012,"periodDesc":"2012","aggrLevel":4,"IsLeaf":0,"rgCode":2,"rgDesc":"Export","rtCode":842,"rtTitle":"USA","rt3ISO":"USA","ptCode":0,"ptTitle":"World","pt3ISO":"WLD","cmdCode":"5989","cmdDescE":"Chemical products and preparations, n.e.s.","qtCode":8,"qtDesc":"Weight in kilograms","TradeQuantity":2044214354,"NetWeight":2044214354,"TradeValue":6057293127,"estCode":0},

alexandersimoes commented 10 years ago

Thanks for the feedback!

Could you tell me exactly what that comtrade API request is fetching as well as each of the parameters. Our data came from doing batch downloads before they had a working API.

When I take the sum TradeValue key of the JSON returned by the URL above I get the following: 1,545,565,185,671 or ~ 1.5 trillion.

mkvp commented 10 years ago

First of all, thank you very much for your attention.

The API call is http://comtrade.un.org/api/get?max=50000&type=C&freq=A&px=S4&ps=2012&r=842&p=0&rg=2&cc=ALL The search is: annual frequency, SITC - REV 4 classification, Year 2012, from USA to the world, trade flow: export and commodity codes: All SITC Rev.4 categories.

As you said, the sum of export trade value is close to 1.5 Trillion in this search. Nevertheless, when I make the same searching at OEC (http://atlas.media.mit.edu/explore/tree_map/sitc/export/usa/all/show/2012/), total exports trade value is $2.59T. If you make a search for categories, you can also find data discrepancies. For instance, the search for exports for code 5989 give a result of 6B in COMTRADE (http://comtrade.un.org/api/get?max=500&type=C&freq=A&px=S4&ps=2012&r=842&p=0&rg=2&cc=5989) versus a result of 36.8B. in OEC (http://atlas.media.mit.edu/explore/tree_map/sitc/export/usa/all/show/2012).

I'm probably missing something.

Thank you again, and contratulations for your great job in OEC!