haasad / EcoInventDownLoader

Download, unpack and import ecoinvent into your brightway2 project in one simple step
MIT License
13 stars 6 forks source link

Can the EI3.8 system model "cut-off EN 15804" be imported via the EIDL? #19

Closed bsteubing closed 2 years ago

bsteubing commented 2 years ago

@haasad question: can we also download the new cut-off EN 15804 database version from ecoinvent 3.8 using the EIDL? I copy @cmutel as we were just discussion this...

haasad commented 2 years ago

As it turns out, downloading this new Allocation, cut-off, EN15804 database works out-of-the-box with eidl:

In [7]: eidl.get_ecoinvent()
ecoinvent username: ***
ecoinvent password: 
logging in to ecoinvent homepage...
login successful!

 available versions:
a 3.8
b 3.7.1
c 3.7
d 3.6
e 3.5
f 3.4
g 3.3
h 3.2
i 3.1
j 3.01
version: a

 system models:
a EN15804
b apos
c consequential
d cutoff
system model: a
database already downloaded

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 59010226 bytes (57 MiB)                         

Extracting archive: /home/adrian/.local/share/EcoInventDownLoader/EN1580438.7z
--
Path = /home/adrian/.local/share/EcoInventDownLoader/EN1580438.7z
Type = 7z
Physical Size = 59010226
Headers Size = 672095
Method = LZMA2:26
Solid = +
Blocks = 1

Everything is Ok                                                            

Folders: 2
Files: 19551
Size:       2643623421
Compressed: 59010226
Extracting XML data from 19531 datasets
Extracted 19531 datasets in 103.54 seconds
No biosphere database present in your current project: cutoff_en_15804
You can run "bw2setup()" if this is a new project. Run it now?
[y]/n y
Creating default biosphere

Applying strategy: normalize_units
Applying strategy: drop_unspecified_subcategories
Applying strategy: ensure_categories_are_tuples
Applied 3 strategies in 0.01 seconds
Writing activities to SQLite3 database:
0% [##############################] 100% | ETA: 00:00:00
Total time elapsed: 00:00:00
Title: Writing activities to SQLite3 database:
  Started: 11/02/2021 17:30:56
  Finished: 11/02/2021 17:30:56
  Total time elapsed: 00:00:00
  CPU %: 71.60
  Memory %: 41.74
Created database: biosphere3
Creating default LCIA methods

Applying strategy: normalize_units
Applying strategy: set_biosphere_type
Applying strategy: fix_ecoinvent_38_lcia_implementation
Applying strategy: drop_unspecified_subcategories
Applying strategy: link_iterable_by_fields
Applied 5 strategies in 0.74 seconds
Wrote 975 LCIA methods with 254388 characterization factors
Creating core data migrations

Applying strategy: normalize_units
Applying strategy: update_ecoinvent_locations
Applying strategy: remove_zero_amount_coproducts
Applying strategy: remove_zero_amount_inputs_with_no_activity
Applying strategy: remove_unnamed_parameters
Applying strategy: es2_assign_only_product_with_amount_as_reference_product
Applying strategy: assign_single_product_as_activity
Applying strategy: create_composite_code
Applying strategy: drop_unspecified_subcategories
Applying strategy: fix_ecoinvent_flows_pre35
Applying strategy: drop_temporary_outdated_biosphere_flows
Applying strategy: link_biosphere_by_flow_uuid
Applying strategy: link_internal_technosphere_by_composite_code
Applying strategy: delete_exchanges_missing_activity
Applying strategy: delete_ghost_exchanges
54 exchanges couldn't be linked and were deleted. See the logfile for details:
    /home/adrian/.cache/Brightway3/log/cutoff_en_15804.577e0307d7561091b07b4f035a2cdd2a/Ecospold2-import-error.plx1Qu.log
Applying strategy: remove_uncertainty_from_negative_loss_exchanges
Applying strategy: fix_unreasonably_high_lognormal_uncertainties
Applying strategy: set_lognormal_loc_value
Applying strategy: convert_activity_parameters_to_list
Applying strategy: add_cpc_classification_from_single_reference_product
Applying strategy: delete_none_synonyms
Applied 21 strategies in 5.57 seconds
19531 datasets
529361 exchanges
0 unlinked exchanges

Write database EN1580438 in project cutoff_en_15804?
[y]/n y
Writing activities to SQLite3 database:
0% [##############################] 100% | ETA: 00:00:00
Total time elapsed: 00:00:39
Title: Writing activities to SQLite3 database:
  Started: 11/02/2021 17:32:12
  Finished: 11/02/2021 17:32:51
  Total time elapsed: 00:00:39
  CPU %: 78.20
  Memory %: 42.65
Created database: EN1580438
haasad commented 2 years ago

However things like this will prevent it from working in the AB: https://github.com/LCA-ActivityBrowser/activity-browser/commit/669027eebdc6dd5d4be4f0b32ebddbd418bc666e

I'm not sure what the intention of this "fix" is, looks to me like an attempted fix for https://github.com/LCA-ActivityBrowser/activity-browser/issues/649 after it was already resolved in eidl.

bsteubing commented 2 years ago

@haasad thanks a lot, I should have directed this question to @nabilahmed739 ... we'll fix this in the AB. Thanks for the check anyway!

haasad commented 2 years ago

If you want I can prepare an MR for the activity-browser to bring the two projects back in sync. Right now there are a lot of eidl methods that the AB "overwrites", so the AB doesn't get the benefit if things are fixed in eidl, leading to issues like https://github.com/LCA-ActivityBrowser/activity-browser/issues/649#issuecomment-947984913

haasad commented 2 years ago

I opened a PR that makes it possible again to download the cut-off EN 15804 system model in the AB. Closing this issue