IndEcol / pymrio

Multi-Regional Input-Output Analysis in Python.
http://pymrio.readthedocs.io/en/latest/
Other
155 stars 71 forks source link

error when trying to parse EORA26 #63

Closed Rash1ka closed 2 years ago

Rash1ka commented 3 years ago

Hello, The command - eora = pymrio.parse_eora26(year=1990, path=eora_storage) shows an error (KeyError: "There is no item named 'labels_Q.txt' in the archive")

Kindly help me out. Thanks a lot

spjuhel commented 3 years ago

Hello, I had the same problem I think, In my case this was not due to pymrio but to the eora archive lacking this file which gives the label for the different variables of the Q table. I had to download it separately from elsewhere on their website (sadly it is down at the moment so I can't give you an url...)

You might try to add this one to the eora26 folder. labels_Q.txt

I can't be certain the labels are the same (I'm working with year 2015), but they probably are.

konstantinstadler commented 3 years ago

ok , I will check that

spjuhel commented 3 years ago

I'm really not sure this is related, and also not sure this is a pymrio problem but I'm having an issue with the VA in the Eora26 MRIO that might come from the parsing. Hopefully this is the right place to post.

The 'Consumption of fixed capital' for France seems incorrect (very small) when comparing to other 'similar' countries (see outputs below) I've tested with different years (2012, 1990) and the gap is also present. I'm relatively new to MRIO analysis so it is possible I'm missing something (such as France having a very low Consumption of fixed capital, but it doesn't seem to be the case in EXIO3) It's also possible the error comes from the raw data, but I find it strange such an error holds for multiple years for France.

import pymrio as pym
eora_storage = './Data/mrio/eora26'
eora = pym.parse_eora26(year=2015, path=eora_storage)

Out : 
/home/sjuhel/.local/lib/python3.8/site-packages/pymrio/tools/ioparser.py:1894: FutureWarning: The default value of regex will change from True to False in a future version.
  labQ["stressor"] = labQ["stressor"].str.replace(r"\s\((.*)\)", "")
/home/sjuhel/.local/lib/python3.8/site-packages/pandas/core/generic.py:4153: PerformanceWarning: dropping on a non-lexsorted multi-index without a level parameter may impact performance.
  obj = obj._drop_axis(labels, axis, level=level, errors=errors)

eora.VA.F['FRA'].sum(axis=1)

Out : 
inputtype      category                        
Primary input  Compensation of employees D.1       1.625900e+09
               Taxes on production D.29            1.779807e+08
               Subsidies on production D.39       -1.545396e+07
               Net operating surplus B.2n          7.991329e+08
               Net mixed income B.3n               6.694586e+06
               Consumption of fixed capital K.1    2.191005e+01     <- here
dtype: float64

eora.VA.F['GBR'].sum(axis=1)

Out : 
inputtype      category                        
Primary input  Compensation of employees D.1       1.474313e+09
               Taxes on production D.29            5.067867e+07
               Subsidies on production D.39       -1.562346e+06
               Net operating surplus B.2n          5.111755e+08
               Net mixed income B.3n               1.757826e+08
               Consumption of fixed capital K.1    3.268967e+08
dtype: float64

eora.VA.F['ESP'].sum(axis=1)
Out : 
inputtype      category                        
Primary input  Compensation of employees D.1       7.902165e+08
               Taxes on production D.29            2.100026e+07
               Subsidies on production D.39       -9.349826e+06
               Net operating surplus B.2n          4.362211e+08
               Net mixed income B.3n               8.775046e+06
               Consumption of fixed capital K.1    1.165287e+08
dtype: float64
konstantinstadler commented 3 years ago

Did you try to reproduce independently from pymrio (either in python or excel or something) - basically I am asking if you find the same in the Eora souce files or if it is connected to the parsing by pymrio?

spjuhel commented 3 years ago

Ok I managed to do that and it appears the problem comes from Eora26. Sorry for bothering you.

akifnu commented 3 years ago

Dear @konstantinstadler,

I am facing the same problem and I think in code is perfect but the zip file lacks the labels for Q or 'labels_Q.txt' file. image

Akif

konstantinstadler commented 3 years ago

If you want, you can search through these tables if you find a new file which contains labels for Q. I never work with Eora but very much open for code contributions to fix that.

spjuhel commented 3 years ago

Hello, The label_Q.txt file can be found in the indices.zip file on this page I also mentionend to the EORA team that it can be confusing that all labels are present in the tables archives with the only exception being this one.

Maybe mentioning this file is missing (but is easily downloaded) in the readthedoc could be an easy fix ?

akifnu commented 3 years ago

I fixed it by just copying the Q file from the indices to the eora26 zip file and it worked fine