brightway-lca / brightway2

Metapackage for brightway2 imports and documentation
https://brightway.dev/
BSD 3-Clause "New" or "Revised" License
100 stars 37 forks source link

Ecospold2 importer issue with versions 2.1, 2.2, 2.4 (ecoinvent 3.6 and ecoinvent 3.8) #59

Closed tjlca closed 1 year ago

tjlca commented 1 year ago

Using a tried and tested code with ecoinvent 3.6 and ecoinvent 3.8 datasets is causing issues with the ecospold reader command.

The issue is with Unlinked exchanges. But we have applied strategies to remove these issues and this code worked fine with Ecoinvent 3.6 perfectly.

Code Snippet -

        ei_38 = bw.SingleOutputEcospold2Importer(ei38_path, base_database, use_mp=False)
        ei_38.apply_strategies()
        ei_38.statistics()
        ei_38.write_database()        
        print('Successfully written '+base_database, flush = True)

Error

    ei_38.write_database()        
  File "/home/tghosh/.conda-envs/premise/lib/python3.8/site-packages/bw2io/importers/base_lci.py", line 269, in write_database
    db.write(existing)
  File "/home/tghosh/.conda-envs/premise/lib/python3.8/site-packages/bw2data/project.py", line 358, in writable_project
    return wrapped(*args, **kwargs)
  File "/home/tghosh/.conda-envs/premise/lib/python3.8/site-packages/bw2data/backends/peewee/database.py", line 260, in write
    self._efficient_write_many_data(data)
  File "/home/tghosh/.conda-envs/premise/lib/python3.8/site-packages/bw2data/backends/peewee/database.py", line 204, in _efficient_write_many_data
    exchanges, activities = self._efficient_write_dataset(
  File "/home/tghosh/.conda-envs/premise/lib/python3.8/site-packages/bw2data/backends/peewee/database.py", line 156, in _efficient_write_dataset
    raise InvalidExchange
bw2data.errors.InvalidExchange

Output -

Readingecoinvent3.6
/scratch/tghosh/liaison2_gcam/jobdirsubmit_ecoinvent_2020_SSP2-Basedb_creatorpeme/data/inputs/ecoinvent/ecoinvent_3.6_cutoff_ecoSpold02/datasets
Extracting XML data from 18121 datasets
Extracted 18121 datasets in 16.26 seconds
Applying strategy: normalize_units
Applying strategy: update_ecoinvent_locations
Applying strategy: remove_zero_amount_coproducts
Applying strategy: remove_zero_amount_inputs_with_no_activity
Applying strategy: remove_unnamed_parameters
Applying strategy: es2_assign_only_product_with_amount_as_reference_product
Applying strategy: assign_single_product_as_activity
Applying strategy: create_composite_code
Applying strategy: drop_unspecified_subcategories
Applying strategy: fix_ecoinvent_flows_pre35
Applying strategy: drop_temporary_outdated_biosphere_flows
Applying strategy: link_biosphere_by_flow_uuid
Applying strategy: link_internal_technosphere_by_composite_code
Applying strategy: delete_exchanges_missing_activity
Applying strategy: delete_ghost_exchanges
Applying strategy: remove_uncertainty_from_negative_loss_exchanges
Applying strategy: fix_unreasonably_high_lognormal_uncertainties
Applying strategy: set_lognormal_loc_value
Applying strategy: convert_activity_parameters_to_list
Applying strategy: add_cpc_classification_from_single_reference_product
Applying strategy: delete_none_synonyms
Applied 21 strategies in 7.29 seconds
18121 datasets
615644 exchanges
2038 unlinked exchanges
  Type biosphere: 17 unique unlinked exchanges
Warning: No valid output stream.
Fri Dec 16 10:57:15 MST 2022
copying result
2 minutes and 31 seconds elapsed.
tjlca commented 1 year ago

bw2io should be downgraded to 0.8.7 to resolve this issue.

bw2io 0.8.8 is only for ecoinvent 3.9