Unit incompatibility error when reporting some scenarios

Jihoon commented 3 years ago

Simple message_ix reporting gives the following error for some sets of models (e.g. ENGAGE, NGFS) but not for some other models (e.g. MESSAGEix-Materials). This error is coming from either fom or vom key, but those incompatible units are not found from corresponding parameters.

Code sample or context


import ixmp as ix
import message_ix
from message_ix.reporting import Reporter

mp = ix.Platform('ixmp_dev')
Sc_ref = message_ix.Scenario(mp, 'ENGAGE_SSP2_v4.1.7', 'EN_NPi2020_1400')
rep = Reporter.from_scenario(Sc_ref)
df = rep.get("message:default")

Expected result

An dfoutput of type pyam.core.IamDataFrame

Problem description

Error (Units incompatible)

(base) C:\WINDOWS\system32>mix-models --url="ixmp://ixmp_dev/ENGAGE_SSP2_v4.1.7/EN_NPi2020_1400" dle report --old_reporting False
C:\ProgramData\Anaconda3\lib\site-packages\sdmx\session.py:13: RuntimeWarning: optional dependency requests_cache is not installed; cache options to Session() have no effect
  RuntimeWarning,
2021-06-28 11:24:05,172  INFO at.ac.iiasa.ixmp.Platform:146 - Welcome to the IX modeling platform!
2021-06-28 11:24:05,174  INFO at.ac.iiasa.ixmp.Platform:147 -  connected to database 'jdbc:oracle:thin:@x8oda.iiasa.ac.at:1521/PIXMP2.iiasa.ac.at' (user: ixmp_dev)...
This Scenario has a solution, use `Scenario.remove_solution()` or `Scenario.clone(..., keep_solution=False)`
ixmp.model.base.initialize_items  This Scenario has a solution, use `Scenario.remove_solution()` or `Scenario.clone(..., keep_solution=False)`
genno.config.units  Replace unit '-' with ''
fix_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
ixmp.reporting.computations.data_for_quantity  fix_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
inv_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
...data_for_quantity  inv_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
emission_factor: mixed units ['tC', '???', 'kg/kWa', '-'] discarded
...data_for_quantity  emission_factor: mixed units ['tC', '???', 'kg/kWa', '-'] discarded
input: mixed units ['GWa', '-', '???'] discarded
...data_for_quantity  input: mixed units ['GWa', '-', '???'] discarded
output: mixed units ['GWa', '-'] discarded
...data_for_quantity  output: mixed units ['GWa', '-'] discarded
numexpr.utils._init_num_threads  NumExpr defaulting to 8 threads.
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\Scripts\mix-models-script.py", line 33, in <module>
    sys.exit(load_entry_point('message-ix-models', 'console_scripts', 'mix-models')())
  File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "h:\mydocuments\message\message_data\message_data\model\dle\__init__.py", line 168, in run_reporting
    report(scenario, old_reporting)
  File "h:\mydocuments\message\message_data\message_data\model\dle\reporting.py", line 192, in report
    df = rep.get("message:default")
  File "C:\ProgramData\Anaconda3\lib\site-packages\genno\core\computer.py", line 407, in get
    raise ComputationError(exc) from None
genno.core.exceptions.ComputationError: computing <tom:nl-t-yv-ya> using:

(<function add at 0x000001B42CF88A68>, 'fom:nl-t-yv-ya', 'vom:nl-t-yv-ya')

Use Computer.describe(...) to trace the computation.

Computation traceback:
  File "C:\ProgramData\Anaconda3\lib\site-packages\genno\computations.py", line 74, in add
    raise ValueError(f"Units '{ref_unit:~}' and '{u:~}' are incompatible")
ValueError: Units '' and 'USD / GWa' are incompatible

Versions

Output of message-ix show-versions

``` ixmp: 3.3.0a0 0188de0 (HEAD -> origin/main) Re-enable py3.9 runs of "pytest" CI workflow message_ix: 3.2.1.dev55+g2107fbc.d20210506 a73cc36 (HEAD -> master, VK/master) Merge pull request #456 from iiasa/issue_437 message_data: 0.1.dev0 6c656e01 (HEAD -> DLE-MESSAGE, Jihoon/DLE-MESSAGE) Add an emission constraint click: 7.1.2 dask: 2021.04.0 graphviz: 0.12 jpype: 1.2.1 … JVM path: C:\Program Files\Java\jdk1.8.0_202\jre\bin\server\jvm.dll openpyxl: 3.0.7 pandas: 1.2.4 pint: 0.14 xarray: 0.17.0 yaml: 5.4.1 iam_units: installed jupyter: installed matplotlib: 3.3.4 plotnine: 0.7.1 pyam: 0.10.0 GAMS: 26.1.0 python: 3.7.10 (default, Feb 26 2021, 13:06:18) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None ```

khaeru commented 3 years ago

Thanks for the report!

Can you do the following for at least (a) a scenario where this error occurs and (b) one where it does not?

for name in ("inv_cost", "fix_cost", "var_cost"):
    print(name, sorted(scen.par(name)["unit"].unique()))

This will help us detect differences in the contents of the scenarios that could be a cause of this error.

khaeru commented 3 years ago

Incidentally: there could be two or more issues and corresponding fixes here:

The scenarios have inconsistent structure and contents—that needs to be fixed in message-ix-models and message_data, either by:
- Generating better scenarios ex-ante / before they are solved.
- Improving the default reporting configuration (global.yaml) for the full model in order to apply proper units ex-post.
message_ix, ixmp, and/or genno are not tolerant of these inconsistent contents. This might be fixed in this package.

GamzeUnlu95 commented 3 years ago

(a) For a scenario that has the error:

inv_cost ['USD/GWa', 'USD/kWa']
fix_cost ['USD/GWa', 'USD/kWa']
var_cost ['USD/GWa']

(b) For a scenario with no error:

inv_cost ['USD/GWa', 'USD/kWa', 't']
fix_cost ['USD/GWa', 'USD/kWa', 't']
var_cost ['USD/GWa', 't']

khaeru commented 3 years ago

Thanks. I'm puzzled why there is no "mixed units discarded" log message for fix_cost (for fom) and var_cost (for vom).

Next, for each scenario:

print(rep.get("fom:nl-t-yv-ya"))
print(rep.get("vom:nl-t-yv-ya"))

Here we're trying to climb back up the chain of calculations to see where the incompatible values originate.

Jihoon commented 3 years ago

Thanks. I'm puzzled why there is no "mixed units discarded" log message for fix_cost (for fom) and var_cost (for vom).

Next, for each scenario:
print(rep.get("fom:nl-t-yv-ya"))
print(rep.get("vom:nl-t-yv-ya"))
Here we're trying to climb back up the chain of calculations to see where the incompatible values originate.

Sorry, it was my bad that I didn't copy the whole error log correctly. I re-pasted the log above.

Jihoon commented 3 years ago

From the conversation with @GamzeUnlu95, we've figured out, for case (b) above, all the units are discarded for the three parameters so that they become compatible with each other. But for case (a), var_cost will still keep 'USD/GWa' causing the incompatibility.

So we were luckily able to use the reporting for our model so far, thanks to the arbitrary addition of 't'.

The most fundamental way to fix it will be to harmonize the units for the cost params to one (MUSD/GWa or USD/kWa), as @GamzeUnlu95 suggests the unit 'USD/GWa' is not correct in the first place. But there will be many other workarounds depending on priorities..

khaeru commented 3 years ago

(b) above, all the units are discarded for the three parameters so that they become compatible with each other. But for case (a), var_cost will still keep 'USD/GWa' causing the incompatibility.

Indeed, I think this is the correct diagnosis.

Another workaround could be to use units/apply/var_cost: "" in the configuration / YAML file to force var_cost to be dimensionless, to the same effect.

The most fundamental way to fix it will be to harmonize the units for the cost params to one (MUSD/GWa or USD/kWa), as @GamzeUnlu95 suggests the unit 'USD/GWa' is not correct in the first place.

Agreed. So indeed fixing the scenario contents is probably in message_data or message-ix-models. We can either transfer this issue, or close it and open a new one there.

However, there is a more fundamental issue that's actually in message_ix or ixmp. I realized it when we first developed the reporting code, but no one has yet run into it:

ixmp parameters, such as var_cost, have units. These are stored, and can be retrieved and used by reporting.
For variables, such as ACT, in reality they do have units, but ixmp was not originally built to handle these.
So we mostly treat these implicitly.
For instance, if you run a coal_ppl and it outputs 100 (the activity, ACT), and its variable cost is 2 USD / kW·a (var_cost). The units of ACT are implicitly kW·a. So the total variable cost is ACT × var_cost [=] kW·a × (USD / kW·a) [=] USD.
It's very reasonable for each (technology, commodity) pair in to have different units for activity. For instance, for a technology that outputs commodity water, it might be natural to denote water in m³, and thus also the ACT. Then the var_cost for this technology should be in units of USD / m³, and USD / kW·a would be a weird choice.
To truly fix this would need deeper changes in the ixmp internals to store and retrieve the units of ACT.
Failing that, we could have a "permanent workaround" in the reporting code, by these steps:
- Split ACT and var_cost, etc. into multiple different quantities—each with consistent units.
- Auto-configure fom, vom, tom etc. separately for each of these.

Writing this down here simply so we are aware of it when doing further development.

Jihoon commented 3 years ago

Another workaround could be to use units/apply/var_cost: "" in the configuration / YAML file to force var_cost to be dimensionless, to the same effect.

A quick question about this workaround. Since @GamzeUnlu95's script is not relying on a yaml file, I've tried this below instead, which didn't resolve the incompatibility as I hoped. Do we need something more?

from ixmp.reporting import configure
configure(units={"apply": {"var_cost": ""}})

I am also curious when units/apply/var_cost is applied correctly, whether it applies to the entire platform or specifically to this model somehow?

khaeru commented 3 years ago

A Reporter object is always needed to run reporting. Some configuration settings are global, others can be set differently on each Reporter instance. See “Global and specific configuration” in the genno docs.

Here's the documentation for how units/apply is handled in ixmp: https://docs.messageix.org/projects/ixmp/en/stable/reporting.html#ixmp.reporting.units (please read).

Finally, note that extra keyword arguments to from_scenario() are forwarded to configure(). So with those points, you should instead do:

from message_ix import Reporter
r = Reporter.from_scenario(scen, units={"apply": {"var_cost": ""}})
r.get(…)

This ensures (per the first link above) that the units setting is retained and applied when data is pulled out of scen.

To directly answer your final question, the configuration is not tied to either (a) a particular ixmp.Platform object nor (b) a particular message_ix.Scenario object. This means that, after the above, you can do this:

# Change the "scenario" key in the already-configured Reporter to point to a different scenario object
# All other described computations remain the same
r.add("scenario", scen2)

# Configuration set above will still apply, but data will be extracted from `scen2`, not `scen`
r.get(…)

iiasa / message_ix