brightway-lca / brightway2-io

Importing and exporting for the Brightway LCA framework
BSD 3-Clause "New" or "Revised" License
26 stars 40 forks source link

Failing EcoInvent 3.9 import #183

Closed tfardet closed 1 year ago

tfardet commented 1 year ago

On Linux, with python 3.10, importing ecoinvent_39_cutoff_lcia_ecoSpold02/datasets (from ecoinvent 3.9_cutoff_lcia_ecoSpold02.7z) fails with ValueError with all versions of bw2io >= 0.8.8 (I did not test older versions).

The error is due to the fact that there is an unrecognized entry tags {http://www.EcoInvent.org/EcoSpold02}impactIndicator that leads to the ValueError in ecospold2.py.

I "fixed" it by adding

    elif exc.tag == "{http://www.EcoInvent.org/EcoSpold02}impactIndicator":
            flow = "impactIndicator"
            is_biosphere = False

but:

  1. I've no idea if that's correct
  2. the import ate 20 GB of RAM (which is not normal as far as I can remember) with use_mp=False in a Jupyter notebook

Should I just ignore these entries instead? I'm not sure why they are here since they don't seem to be supported and my colleagues running Windows do not seem to encounter that issue with the same bw/bw2io versions... (and the same EcoInvent database, obviously).

tfardet commented 1 year ago

Apparently the import did not work properly... For instance the market for electricity, medium voltage' (kilowatt hour, FR, None) entry (code 16f2487031310001380a0524ee7bb120) has only one exchange with itself and therefore does not appear in the LCA characterized inventory, leading to incorrect results.

I tried again with is_biosphere = True but that led to invalid exchanges and I could not write the database.

EDIT: I also tried if ("parameter" not in exc.tag and "impactIndicator" not in exc.tag) on line 306, this makes things very fast and suppresses the RAM explosion but it does not lead to a valid database (market electricity is still lacking the relevant flows)

tfardet commented 1 year ago

Associated issue raised in ActivityBrowser: https://github.com/LCA-ActivityBrowser/activity-browser/issues/247

tfardet commented 1 year ago

Figure it out: the file that needs to be used is ecoinvent 3.9_cutoff_ecoSpold02.7z and not the _lci(a) variant, these are not supported.