Error occurs when runing MESSAGEix-GLOBIOM

Pfyuan77 commented 8 months ago

Code sample or context

<--


import pandas as pd
import ixmp
import message_ix

from message_ix.utils import make_df

data_file = "MESSAGEix-GLOBIOM_1.1_R11_no-policy_baseline.xlsx"

# Loading Modeling platform
mp = ixmp.Platform()

# Creating a new, empty scenario
scenario = message_ix.Scenario(
    mp, model = "Global_SSP2", scenario = "baseline1", version = "new"
)

scenario.read_excel(data_file, add_units = False, init_items = True)

-->

Expected result

<-- read the global scenario of MESSAGEix-GLOBIOM that was writen in the "MESSAGEix-GLOBIOM_1.1_R11_no-policy_baseline.xlsx", then use the scenario to do some simulation -->

Problem description

<--

ValueError                                Traceback (most recent call last)
Cell In[5], line 1
1 scenario.read_excel(data_file, add_units = False, init_items = True)

File ~\anaconda3\lib\site-packages\ixmp\core\scenario.py:927, in Scenario.read_excel(self, path, add_units, init_items, commit_steps)
    901 def read_excel(
    902     self,
    903     path: PathLike,
   (...)
    906     commit_steps: bool = False,
    907 ) -> None:
    908     """Read a Microsoft Excel file into the Scenario.
    909 
    910     Parameters
   (...)
    925     to_excel
    926     """
--> 927     self.platform._backend.read_file(
    928         Path(path),
    929         ItemType.MODEL,
    930         filters=dict(scenario=self),
    931         add_units=add_units,
    932         init_items=init_items,
    933         commit_steps=commit_steps,
    934     )

File ~\anaconda3\lib\site-packages\ixmp\backend\jdbc.py:505, in JDBCBackend.read_file(self, path, item_type, **kwargs)
    478 """Read Platform, TimeSeries, or Scenario data from file.
    479 
    480 JDBCBackend supports reading from:
   (...)
    501 .Backend.read_file
    502 """
    503 try:
    504     # Call the default implementation, e.g. for .xlsx
--> 505     super().read_file(path, item_type, **kwargs)
    506 except NotImplementedError:
    507     pass

File ~\anaconda3\lib\site-packages\ixmp\backend\base.py:431, in Backend.read_file(self, path, item_type, **kwargs)
    429     ts_read_file(s, path, **kwargs)
    430 elif path.suffix == ".xlsx" and item_type is ItemType.MODEL and s:
--> 431     s_read_excel(self, s, path, **kwargs)
    432 else:
    433     raise NotImplementedError

File ~\anaconda3\lib\site-packages\ixmp\backend\io.py:308, in s_read_excel(be, s, path, add_units, init_items, commit_steps)
    303 if not len(idx_sets):
    304     # No index sets -> scalar parameter; must supply empty 'key' column
    305     # for add_par()
    306     df["key"] = None
--> 308 s.add_par(name, df)
    310 # Commit after every parameter
    311 maybe_commit(s, commit_steps, f"Loaded {ix_type} {repr(name)} from {path}")

File ~\anaconda3\lib\site-packages\ixmp\core\scenario.py:536, in Scenario.add_par(self, name, key_or_data, value, unit, comment)
    530 elements = map(
    531     lambda e: (e.key, e.value, e.unit, e.comment),
    532     data.astype(types).itertuples(),
    533 )
    535 # Store
--> 536 self._backend("item_set_elements", "par", name, elements)

File ~\anaconda3\lib\site-packages\ixmp\core\timeseries.py:108, in TimeSeries._backend(self, method, *args, **kwargs)
    102 def _backend(self, method, *args, **kwargs):
    103     """Convenience for calling *method* on the backend.
    104 
    105     The weak reference to the Platform object is used, if the Platform is still
    106     alive.
    107     """
--> 108     return self.platform._backend(self, method, *args, **kwargs)

File ~\anaconda3\lib\site-packages\ixmp\backend\base.py:54, in Backend.__call__(self, obj, method, *args, **kwargs)
     48 def __call__(self, obj, method, *args, **kwargs):
     49     """Call the backend method `method` for `obj`.
     50 
     51     The class attribute obj._backend_prefix is used to determine a prefix for the
     52     method name, e.g. 'ts_{method}'.
     53     """
---> 54     return getattr(self, method)(obj, *args, **kwargs)

File ~\anaconda3\lib\site-packages\ixmp\backend\jdbc.py:1059, in JDBCBackend.item_set_elements(self, s, type, name, elements)
   1056 except java.IxException as e:
   1057     if any(s in e.args[0] for s in ("does not have an element", "The unit")):
   1058         # Re-raise as Python ValueError
-> 1059         raise ValueError(e.args[0]) from None
   1060     elif "cannot be edited" in e.args[0]:
   1061         raise RuntimeError(e.args[0])
ValueError: The unit 'USD_2005/t ' does not exist in the database!

-->

<-- When I use

existing_units = mp.units()
print(existing_units)

I find that the unit 'USD_2005/t' is already exist in the database, but the error still exists. --> <-- The details are also shown below: 微信图片_20240113173220 微信图片_20240113173224 微信图片_20240113173227 -->

Versions

Output of message-ix show-versions

``` ```

khaeru commented 8 months ago

Hi @Pfyuan77—thanks for filing this issue and for your interest in MESSAGEix.

A few points:

You don't say where you obtained the data_file mentioned in your first code snippet. From the file name, I assume/guess it is the MESSAGEix-GLOBIOM snapshot from https://doi.org/10.5281/zenodo.5793870. Please say if this is not the case.
To be clear, the current repository is for MESSAGEix—the generic model framework—only. For specific models of the MESSAGEix-GLOBIOM family, the repository is iiasa/message-ix-models, and issues should be filed there.
Looking at the docs for the latter, particularly the docs about snapshots at https://docs.messageix.org/projects/models/en/latest/api/model-snapshot.html#usage, you can read a relevant note:

For snapshot 0, contrary to the description of the Zenodo item, the file cannot be loaded using [Scenario.read_excel()](). This limitation will be fixed in subsequent snapshots.

For further information, see iiasa/message-ix-models#102, which shows that the issue is due to the "USD_2005/t " vs. "USD_2005/t" units mismatch. There have not been any subsequent snapshots (such as new versions of that existing one) prepared by my colleagues, so this remains the case.
The documentation page linked above also describes alternate code that works around the offending unit strings whilst loading the data. Please read the documentation and try it out; if it does not work, please file an issue on message-ix-models.

Pfyuan77 commented 8 months ago

Deer khaeru, Thanks for replying. I will check "Fetch and load public base model snapshot message-ix-models#102" for further information.

sincerely, Yuan, Shandong University.

	元鹏飞

@. | ---- Replied Message ---- | From | Paul Natsuo @.> | | Date | 1/15/2024 16:38 | | To | @.> | | Cc | Pengfei @.> , @.***> | | Subject | Re: [iiasa/message_ix] Error occurs when runing MESSAGEix-GLOBIOM (Issue #780) |

Closed #780 as not planned.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

iiasa / message_ix