example notebook issues

[marc-vdm]

The example is breaking on reading the datapackage when I try to run it straight in python:

>>> from pathways import Pathways
>>> import numpy as np
>>> p = Pathways(
... datapackage="datapackage_sample/datapackage.json",
... debug=True)
Invalid datapackage: Descriptor validation error: {'path': 'mapping/mapping.yaml', 'profile': 'data-resource', 'name': 'mapping', 'format': 'yaml', 'mediatype': 'text/yaml', 'encoding': 'utf-8'} is not valid under any of the given schemas at "resources/33" in descriptor and at "properties/resources/items/oneOf" in profile
Invalid datapackage: Descriptor validation error: 'data-resource' is not one of ['tabular-data-resource'] at "resources/33/profile" in descriptor and at "properties/resources/items/properties/profile/enum" in profile
Invalid datapackage: Descriptor validation error: {'path': 'classifications/classifications.yaml', 'profile': 'data-resource', 'name': 'classifications', 'format': 'yaml', 'mediatype': 'text/yaml', 'encoding': 'utf-8'} is not valid under any of the given schemas at "resources/34" in descriptor and at "properties/resources/items/oneOf" in profile
Invalid datapackage: Descriptor validation error: 'data-resource' is not one of ['tabular-data-resource'] at "resources/34/profile" in descriptor and at "properties/resources/items/properties/profile/enum" in profile
Log file: /home/marc/.local/state/pathways/log/pathways.log

I have the example folder straight from the repo with git clone.

[romainsacchi]

Yes, these are warnings only. It complains that some resources (the YAML files) in the data package do not comply with the tabular data format. It does not prevent the code execution, but yes, I should fix/silence that.

[marc-vdm]

Right, thanks.

Two minor other points:

1. The decryption key is in the notebook
2. Typo in first sentence of 'avoiding double counting' section:

When working with IAM/ESM, the focus s in the total quantity

[romainsacchi]

OK, fixed. I'll fix the warnings later today.

[marc-vdm]

The calculation is also not working, I expect due to a specific version of bw2calc not being specified:

>>> p.calculate(
...     methods=[ # <-- we list LCIA methods here
...         "EF v3.1 EN15804 - climate change - global warming potential (GWP100)",
...         'IPCC 2021 - climate change: fossil - global warming potential (GWP20)'
...     ],
...     regions=["EU",], # <-- the regions to conduct the assessment for
...     scenarios=[
...         "Scenario A",
...         "Scenario B",
...     ],
...     variables=[
...         "technology A",
...     ],
...     years=[
...         2020,
...         2030,
...         2040,
...         2050
...     ],
...     use_distributions=500, # <-- if >0, Monte Carlo analysis
...     multiprocessing=True # parallelize calculations for faster results
... )
Calculating LCA results for some model...
--- Calculating LCA results for Scenario A...
------ Calculating LCA results for 2020...
------ Calculating LCA results for 2030...
------ Calculating LCA results for 2040...
------ Calculating LCA results for 2050...
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/marc/.conda/envs/pw/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/marc/.conda/envs/pw/lib/python3.11/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
           ^^^^^^^^^^^^^^^^
  File "/home/marc/.conda/envs/pw/lib/python3.11/site-packages/pathways/lca.py", line 632, in _calculate_year
    lca = bc.MultiLCA(
          ^^^^^^^^^^^^
TypeError: MultiLCA.__init__() got an unexpected keyword argument 'demands'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/marc/.conda/envs/pw/lib/python3.11/site-packages/pathways/pathways.py", line 478, in calculate
    years, p.map(_calculate_year, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marc/.conda/envs/pw/lib/python3.11/multiprocessing/pool.py", line 367, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marc/.conda/envs/pw/lib/python3.11/multiprocessing/pool.py", line 774, in get
    raise self._value
TypeError: MultiLCA.__init__() got an unexpected keyword argument 'demands'
>>> 
>>> (pw) marc@framework7640u:~/Downloads/temp/pathways/example/datapackage_sample$ cd mapping/
  File "<stdin>", line 1
    (pw) marc@framework7640u:~/Downloads/temp/pathways/example/datapackage_sample$ cd mapping/
         ^^^^
SyntaxError: invalid syntax
>>> (pw) marc@framework7640u:~/Downloads/temp/pathways/example/datapackage_sample/mapping$ la
  File "<stdin>", line 1
    (pw) marc@framework7640u:~/Downloads/temp/pathways/example/datapackage_sample/mapping$ la
         ^^^^
SyntaxError: invalid syntax
>>> mapping.yaml
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'mapping' is not defined

My bw setup is:

(pw) marc@framework7640u:~/Downloads/temp/pathways$ conda list bw
# packages in environment at /home/marc/.conda/envs/pw:
#
# Name                    Version                   Build  Channel
bw2calc                   1.8.2           py311h38be061_0    conda-forge
bw2data                   3.6.6              pyhd8ed1ab_0    conda-forge
bw2io                     0.8.12             pyhd8ed1ab_0    conda-forge
bw2parameters             1.1.0              pyhd8ed1ab_0    conda-forge
bw_migrations             0.2                pyhd8ed1ab_0    conda-forge
bw_processing             1.0                pyhd8ed1ab_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge

[romainsacchi]

OK, fixed in https://github.com/polca/pathways/commit/02a702f62dbab8d6163fa54da673a625e35b1efe. I forgot to specify versions for bw2calc in the conda recipe file. It should have installed bw2calc from bw25.

[marc-vdm]

Thanks, will try again soon

[romainsacchi]

Datapackage validation issues silenced in https://github.com/polca/pathways/commit/dbc3ecee61b97ce85080ede433ec1f0f1cf86e23 But only those caused by the presence of the YAML (other validation errors will still be printed), which do not have a corresponding profile schema in data packages because they are not tabular data file formats.

[marc-vdm]

That doesn't work in conda, when it pulls 2024.10.30 is still installs bw versions as listed above. The dev versions are considered 'lower' by conda. What does work is bw2calc = 2.0.dev23 etc.

I now installed the correct versions with conda create -n pw -c conda-forge -c romainsacchi -c cmutel pathways bw2calc=2.0.dev23 bw2data=4.0.dev59

[romainsacchi]

OK, thanks. I'll change that.