database writing failed due to unlinked exchanges in ecoinvent 3.10

[IsaacZhucheng]

Hi! Ecoinvent just released their 3.10 version and I am trying to use this version.

projects.set_current("test2")

# set up db
#del databases["biosphere3"]
bw2setup()

db_name = "ei3_10"
# check if the ei db already imported
if db_name in databases:
    print("Ecoinvent DB is ready to use in cache", "\n")
    # Otherwise load it
else:
    db = SingleOutputEcospold2Importer(EI_DB_PATH, db_name, use_mp=False)
    db.apply_strategies()
    db.statistics()
    db.write_database()

I tried to update bw2 to latest version, and delete biosphere3 db and re-setup them. But I constantly got unlinked exchanges issue:

23523 datasets
743409 exchanges
4034 unlinked exchanges
  Type biosphere: 62 unique unlinked exchanges
Warning: No valid output stream

Here is one of the exchanges that is unlinked:

{'flow': '55b24735-f967-4cdd-8620-db67af975992', 'type': 'biosphere', 'name': 'Acenaphthylene', 'classifications': {'CPC': []}, 'production volume': 0.0, 'properties': {}, 'unit': 'kilogram', 'amount': 1.77280105219015e-08, 'uncertainty type': 0, 'loc': 1.77280105219015e-08}

Wondering if there is any fix for this?

[cmutel]

Fixed in 0.8.12

[IsaacZhucheng]

Hi @cmutel Sorry to bother again. I might not be doing this the correct way. I update bw2io and create a new proejct, re-use the codes above but getting the same error. Should I use your import_ecoinvent_release instead? My brightway2 is 2.4.4 (it says it is not compatible with 0.8.12 though, but I didn't find a newer version)

[tngTUDOR]

This seems to have been fixed for legacy, but I can reproduce the error with bw25:

In [1]: import bw2data as bd

In [2]: import bw2io as bi

In [3]: bd.__version__
Out[3]: (4, 0, 'DEV33')

In [4]: bi.__version__
Out[4]: (0, 9, 'DEV26')

In [13]: importer = bi.SingleOutputEcospold2Importer('/home/upolig/Documents/dbs/ecoinvent/3.10/datasets/', 'ecoinvent-3.10-cutoff')
Extracting XML data from 23523 datasets
Extracted 23523 datasets in 38.39 seconds

In [14]: importer.apply_strategies()
Applying strategy: normalize_units
Applying strategy: update_ecoinvent_locations
Applying strategy: remove_zero_amount_coproducts
Applying strategy: remove_zero_amount_inputs_with_no_activity
Applying strategy: remove_unnamed_parameters
Applying strategy: es2_assign_only_product_with_amount_as_reference_product
Applying strategy: assign_single_product_as_activity
Applying strategy: create_composite_code
Applying strategy: drop_unspecified_subcategories
Applying strategy: fix_ecoinvent_flows_pre35
Applying strategy: drop_temporary_outdated_biosphere_flows
Applying strategy: link_biosphere_by_flow_uuid
Applying strategy: link_internal_technosphere_by_composite_code
Applying strategy: delete_exchanges_missing_activity
Applying strategy: delete_ghost_exchanges
Applying strategy: remove_uncertainty_from_negative_loss_exchanges
Applying strategy: fix_unreasonably_high_lognormal_uncertainties
Applying strategy: convert_activity_parameters_to_list
Applying strategy: add_cpc_classification_from_single_reference_product
Applying strategy: delete_none_synonyms
Applying strategy: update_social_flows_in_older_consequential
Applying strategy: set_lognormal_loc_value
Applied 22 strategies in 3.55 seconds

In [15]: importer.statistics()
23523 datasets
743409 exchanges
4034 unlinked exchanges
  Type biosphere: 62 unique unlinked exchanges
Out[15]: (23523, 743409, 4034)

[tngTUDOR]

As of bw2io version 0.9.DEV26,

Importing ei 3.10 with ecoinvent_interface ✅

This works for both, legacy and bw25.

For the "legacy" bw2 io, you need to make sure you have bwio version 0.8.12 exactly, install ecoinvent_interface and follow the Howto instructions at the bottom of this comment.

Importing ei 3.10 with SingleOutputEcospold2Importer 🕳️

• This requires a biosphere db to exist already in the project (this can be done with bw2setup())
• the bw2setup() function will create a biosphere db compatible with ecoinvent 3.9: from this file, at this point in code:

def create_default_biosphere3(overwrite=False):
    from .importers import Ecospold2BiosphereImporter

    eb = Ecospold2BiosphereImporter()
    eb.apply_strategies()
    eb.write_database(overwrite=overwrite)

(because the Ecospold2BiosphereImporter() class will take as default "3.9" as version, and because there's also no "3.10" file.

Hence, when trying to use the SingleOutputEcospold2Importer to import ecoinvent 3.10, there are unlinked biosphere exchanges. so this leads to 🕳️

What to do ?

The current solution to add ecoinvent 3.10 to a project, using brightway25 is to use the ecoinvent_interface feature of bw2io:

Prerequisite

install ecoinvent_interface:

pip install ecoinvent_interface

or

conda install -c conda-forge ecoinvent_interface

Howto

import bw2data as bd
import bw2io as bi

bd.projects.set_current('☕-rulez')
bi.import_ecoinvent_release(version="3.10", \ 
    system_model="cutoff", username=( ˘ ³˘)ノ°ﾟº❍｡, password=⚃)