Unable to establish real connection with iiasa.

[hbshrestha]

I had recently upgraded pyam-iamc to 1.3.1. Before that, the connection with scenario explorer hosted by IIASA was quite fast. However, I encountered some weird errors lately.

The versions of pandas, matplotlib, and numpy I have are 1.4.1, 3.5.1, and 1.22.2 respectively. I am able to install pyam-iamc in my jupyter notebook terminal. The version is 1.3.1. However, when I import it in jupyter notebook and check the version, it says 1.0.0.

I found this weird. I googled and realized through this issue Installation: No module named 'pyam' (and further connected issues) that pyam and pyam-iamc are two different packages. I tried uninstalling pyam in the terminal, but it says

WARNING: Skipping pyam as it is not installed

, while I am still able to import pyam in the notebook.

I ran the following lines

conn = pyam.iiasa.Connection()
conn.valid_connections

It took me 150 seconds to execute it, and I also see iamc15 as one of the valid connections.

I am able to use pyam.IamDataFrame(file) on a xlsx file that is in my local system.

But, when I try to read data from scenario explorer, for example using

df = pyam.read_iiasa(
    'iamc15',
    model='MESSAGEix*',
    variable=['Emissions|CO2', 'Primary Energy|Coal'],
    region='World',
    meta=['category'])

it takes a lot of time to load, and I get the following error message at the end: TypeError: Expected file path name or file-like object, got <class 'bytes'> type I tried uninstalling the pyam-iamc 1.3.1 version, and using previous versions like 1.2.0, however, the result is still the same.

I also tried installing pyam in a separate environment, I got the new version imported there, however, I am still getting the TypeError. I wonder if this is due to the version of pandas I am using.

I also tried installing the new release using pip install pyam-1.3.1.zip, however, I got

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Could you help me diagnose and solve this issue?

[danielhuppmann]

Are you working in a conda environment using Jupyter notebooks? I had a similar issue - if jupyter is not installed in the conda environment, it resolves to the base environment. And the error message indeed looks similar to what I saw with pandas 1.4 prior to commit https://github.com/IAMconsortium/pyam/commit/b9a3efed359ba8d5c428a5f0bf825ccd5c2f27f5.

And yes, it was stupid but not quite possible to change anymore - the pyam package is distributed on pypi as "pyam-iamc", which causes some headaches.

[hbshrestha]

Thank you for the prompt response Daniel. Yes, I am working in a conda environment using Jupyter notebooks. I also tried installing old pandas version in the environment. But the TypeError seems to persist.

Could you suggest what I could do to avoid this error?

[danielhuppmann]

Please be sure to do

pip install jupyter

in the environment, then restart the terminal.

[hbshrestha]

Thank you Daniel. I am not getting the TypeError now and this worked.

However, I am not sure, why it has become too slow for me to load data from iiasa currently. I wonder if this is also the case for others. This was not the case earlier.

For example, just to run the following lines of code

df = pyam.read_iiasa(
    'iamc15',
    model='MESSAGEix*',
    variable=['Emissions|CO2', 'Primary Energy|Coal'],
    region='World',
    meta=['category']
)

168 rows * 7 columns

it took me 530 seconds nearly 9 minutes. I am pretty sure it was much faster before for me.

[danielhuppmann]

My guess is that this is an issue with the IIASA internet connection speed or server load right now. There should not have been any changes to the infrastructure or the package implementation that would explain the slow response.

[hbshrestha]

I see. Thank you very much for your kind response!