Unlinked biosphere exchange

[romainsacchi]

Hi,

I tried to run the basic premise code but I got the following error:

"bw2data.errors.UnknownObject: Exchange between ('biosphere3', '55b0bc94-7f43-4930-ad57-25af3f892757') and ('ecoinvent_image_SSP2-Base_2050', '0000bba863db4046be6e0ed85522e163') is invalid - one of these objects is unknown (i.e. doesn't exist as a process dataset)"

Do you have any hints what could cause that error?

Code:

      from premise import *
      import bw2data
      import os

      from co.paths import BASE_DP

      bw2data.projects.set_current('premise')

      ndb = NewDatabase(
          scenarios=[
              {'model': 'image', 'pathway': 'SSP2-Base', 'year': 2050},
              {'model': 'remind', 'pathway': 'SSP2-Base', 'year': 2050},
          ],
          source_type='ecospold',
          source_file_path=os.path.join(BASE_DP, 'inputs', 'ecoinvent 3.6_cut-off_ecoSpold02', 'datasets'),
          source_version='3.6',
          key='<decryption key>',
          keep_uncertainty_data=False
      )

      ndb.update_electricity()
      ndb.update_cement()
      ndb.update_steel()
      ndb.update_fuels()
      ndb.update_two_wheelers()
      ndb.update_cars()
      ndb.update_trucks()
      ndb.update_buses()

      ndb.write_db_to_brightway()
      ndb.generate_scenario_report()

Versions: python: 3.11.3 premise: 1.3.9 brightway: 2.4.4 ecoinvent: 3.6 cut-off (ecoSpold2)

[romainsacchi]

I would start by updating premise (latest version 1.8.0). Then, if you use ecoinvent 3.6, you should probably use bw2io 0.8.7 and do

import bw2io
bw2io.create_default_biosphere3(overwrite=True)

to update your biosphere3 database.

Let me know.

[pwierzgala]

Step 1

I see that in the github the latest released premise version is 1.8.0.dev but when I run: pip install premise==1.8.0 I get the following error:

ERROR: Could not find a version that satisfies the requirement premise==1.8.0 (from versions: 1.1.6, 1.1.7, 1.1.8, 1.1.9, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.3.9)
ERROR: No matching distribution found for premise==1.8.0

I workaround it with pip install https://github.com/polca/premise/archive/refs/tags/v.1.8.0.dev1.zip

Step 2

It turned out that I can't install bw2io==0.8.7 with premise==1.8.0.dev because "premise 1.8.0.dev1 requires bw2io>=0.8.10". The last premise version using bw2io==0.8.7 is 1.7.1. It also requires python '<3.11,>=3.9 so I downgraded python to version 3.9.13.

Step 3

The code from the fist post was modified by adding line: bw2io.create_default_biosphere3(overwrite=True) in a line below brightway project creation:

bw2data.projects.set_current('premise2')
bw2io.create_default_biosphere3(overwrite=True)  # new line

Now, the error is:

KeyError: "Can't find linking activity for exchange:\n{'amount': 134000.0,\n 'comment': 'Air contact module',\n 'location': 'RoW',\n 'name': 'market for concrete, normal strength',\n 'reference product': 'concrete, normal strength',\n 'type': 'technosphere',\n 'unit': 'cubic meter'}"

raised by the line:

ndb.write_db_to_brightway()

Step 4

I tried to run exactly the same code again to double check if I'll get the same error but instead of the previous one I got:

peewee.OperationalError: cannot VACUUM from within a transaction

raise by the line:

bw2io.create_default_biosphere3(overwrite=True)

When I ran it for the third time I got the error from the "step 2". After a couple runs more it looks that errors from step 2 and step 3 appear in cycles.

Versions

python: 3.9.13 premise: 1.7.1 bw2io: 0.8.7 ecoinvent: 3.6 cut-off (ecoSpold2)

[romainsacchi]

@pwierzgala in a few minutes, 1.8.0 will be released. I suggest to install it in a new conda environment. Let me know.

[cmutel]

FWIW, according to my files 55b0bc94-7f43-4930-ad57-25af3f892757 was in 3.6 and is still present in 3.9.1, so I am not sure the version of the ecoinvent flowlist should play any role in this error.

[pwierzgala]

@romainsacchi Thanks for the updates. The problem is solved but some actions on your side my be required. Let me explain.

Initial information

[1] The institution I work for uses ecoinvent 3.6. The brightway documentation says that ecoinvent 3.6 should be loaded with bw2io==0.8.7. The last premise version which accepts bw2io==0.8.7 is 1.7.1. Anyways, I decided to test premise==1.8.0 with ecoinvent 3.9 just to have any working code. [2] At some point I realized that I can't repeat some of my tests and it seemed that it's because I used the same brightway project name for all test. I started to manually remove all brightway project files after each test. After that I stopped experiencing not repeatable tests. However, I can't confirm that it was the reason because maybe I just started to be more careful in noting results of tests.

Ecoinvent 3.9 cut-off

Everything worked without surprises. The brightway documentation specifies that for ecoinvent>3.8 version of bw2io must be >0.8.7. The latest version (1.8.0) of premise works both with brightway2 and brightway25. The table below contains configurations I've tested. All of them worked.

brightway	bw2io	premise	package manager	python
2.5 (1.0.6)	0.9.dev23	1.8.0	conda	3.9.13
2.5 (1.0.6)	0.9.dev23	1.8.0	pip	3.9.13
2 (2.4.4)	0.8.10	1.8.0	pip	3.9.13
2 (2.4.4)	0.8.8	1.8.0	pip	3.9.13
2 (2.4.4)	0.8.10	1.8.0	pip	3.9.13
2 (2.4.4)	0.8.10	1.7.9	pip	3.9.13

Ecoinvent 3.6 cout-off

The brightway documentation specifies that for ecoinvent<=3.8 version of bw2io must be =0.8.7. The latest brightway2 version which works with bw2io==0.8.7 is 2.4.3 and the latest premise version is 1.7.1. The following is the message from pip describing constraints of package dependencies.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
brightway2 2.4.4 requires bw2io==0.8.10, but you have bw2io 0.8.7 which is incompatible.
premise 1.8.0 requires bw2io>=0.8.10, but you have bw2io 0.8.7 which is incompatible.

The following configuration should be the latest one which works with ecoinvent 3.6.	brightway	bw2io	premise	package manager	python
2 (2.4.3)	0.8.7	1.7.1	pip	3.9.13

but the script raises the following error:

KeyError: "Can't find linking activity for exchange:\n{'amount': 134000.0,\n 'comment': 'Air contact module',\n 'location': 'RoW',\n 'name': 'market for concrete, normal strength',\n 'reference product': 'concrete, normal strength',\n 'type': 'technosphere',\n 'unit': 'cubic meter'}"

To fix this I had to ignore dependency requirements raised by premise and use its higher version. Also, I could ignore dependency requirement raised by brightway2==2.4.4 but it's not relevant in this issue. Configurations that I've tested and worked for me are listed below.	brightway	bw2io	premise	package manager	python
2 (2.4.4)	0.8.7	1.8.0	pip	3.9.13
2 (2.4.3)	0.8.7	1.8.0	pip	3.9.13
2 (2.3)	0.8.7	1.8.0	pip	3.9.13
2 (2.3)	0.8.7	1.7.9	pip	3.9.13

Conclusions

[1] Dependency requirements raised by premise are misleading. [2] I have doubts about changing package requirements when only PATCH number is increased. Required bw2io version changes between premise 1.7.1 and 1.7.2. The last number should be increased only to apply bug fixes that are backwards compatible. When I update premise 1.7.1 to 1.7.2 it automatically updates bw2io 0.8.7 to 0.8.10 which brakes backwards compatibility because bw2io==0.8.10 causes my code to stop working.

[romainsacchi]

@pwierzgala thanks for the remark. I'll make sure to make a major version increment in the future when it breaks backward compatibility.