polca / premise

Coupling Integrated Assessment Models output with Life Cycle Assessment.
BSD 3-Clause "New" or "Revised" License
117 stars 48 forks source link

Error when writing database to Brightway #93

Closed felipecampos1994 closed 1 year ago

felipecampos1994 commented 1 year ago

Hi!

Does anybody happen to know why is the error in the attached file happening after wiriting an updated database to Brightway? I'm using the function dba1.write_db_to_brightway().

Apparently, it does write it (I'm not sure if correctly), so I don't even know if such error implies any practical issues.

Thanks in advanced.

image

narierinke commented 1 year ago

Hi! Yesterday I was having another error, but also when writing to brightway. In my case, it didn't write at all. Couldn't fix yet. error

romainsacchi commented 1 year ago

Hi both,

could you check you are using wurst 0.3.3? If the version of wurst is 0.3.4, you need to downgrade. I'll pin the correct wurst version in the next release.

felipecampos1994 commented 1 year ago

Thanks Romain,

I downgraded to wurst 0.3.3 (I was indeed usinng 0.3.4) but the same error keeps happening.

romainsacchi commented 1 year ago

Hi @felipecampos1994 ,

did you restart the kernel of your notebook before retrying? @narierinke is the issue still here for you too?

felipecampos1994 commented 1 year ago

I was pretty sure I did, but anyways I just tried again. Error is still there.

The good thing is that apparently this error happens right after the first database is written to brightway. Writing them one at the time apparently worked (not sure if entirely right), I have access to them from the activity browser.

romainsacchi commented 1 year ago

Ok, I will release premise 1.4.0 very shortly (maybe even today). That should fix your issue. Will let you know when out.

felipecampos1994 commented 1 year ago

Thanks a lot!

romainsacchi commented 1 year ago

Hey @narierinke @felipecampos1994 , premise v.1.4.0 will be available in a few minutes. Please try it out and let me know if it fixes your issue. Also, do not forget to clear your cache before clear_cache().

felipecampos1994 commented 1 year ago

maybe a silly question...how do i get access to such update? The conda environment that I'm using doesn't seem to be identifying it anywhere

romainsacchi commented 1 year ago

@felipecampos1994 you can uninstall premise and ask for the latest version

pip uninstall premise
pip install premise==1.4.0
felipecampos1994 commented 1 year ago

image

Hi Romain,

Now this is what pops up. I also made sure cache was clear before running the script.

romainsacchi commented 1 year ago

Hi @felipecampos1994 ,

there's not much info given here. Can you print the whole error? Also, can you tell me which scenarios you ran, and which ecoinvent database you use?

felipecampos1994 commented 1 year ago

Yes, sorry. I thought the original screenwhot was wider. I'm running this tryout with the scenario {"model":"remind", "pathway":"SSP2-PkBudg500", "year":2030}, using the ecoinvent 3.8 database.

The error happens again when writing the updated premise database to brightway (after running the dba.write_db_to_brightway() function). Previously, I have updated all exchanges including two-wheelers, cars and buses. The full log is:


KeyError Traceback (most recent call last) File ~\Anaconda3\envs\ab\lib\site-packages\wurst\linking.py:28, in link_internal(data, fields) 27 try: ---> 28 exc["input"] = products[get_tuple(exc)] 29 except KeyError:

KeyError: ('market for internal combustion engine, passenger car', 'internal combustion engine, for passenger car', 'GLO', 'kilogram')

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last) Cell In [8], line 1 ----> 1 dba.write_db_to_brightway()

File ~\Anaconda3\envs\ab\lib\site-packages\premise\ecoinvent_modification.py:1004, in NewDatabase.write_db_to_brightway(self, name) 999 print(f"Prepare database {scen + 1}.") 1000 scenario["database"], cache = prepare_db_for_export( 1001 scenario, cache=cache, name=name[scen] 1002 ) -> 1004 write_brightway2_database( 1005 scenario["database"], 1006 name[scen], 1007 )

File ~\Anaconda3\envs\ab\lib\site-packages\premise\utils.py:753, in write_brightway2_database(data, name) 750 ds["exchanges"] = [clean_up(exc) for exc in ds["exchanges"]] 752 change_db_name(data, name) --> 753 link_internal(data) 754 check_internal_linking(data) 755 check_duplicate_codes(data)

File ~\Anaconda3\envs\ab\lib\site-packages\wurst\linking.py:30, in link_internal(data, fields) 28 exc["input"] = products[get_tuple(exc)] 29 except KeyError: ---> 30 raise KeyError( 31 "Can't find linking activity for exchange:\n{}".format(pformat(exc)) 32 ) 33 return data

KeyError: "Can't find linking activity for exchange:\n{'amount': 297.84083,\n 'location': 'GLO',\n 'name': 'market for internal combustion engine, passenger car',\n 'product': 'internal combustion engine, for passenger car',\n 'type': 'technosphere',\n 'unit': 'kilogram'}"

romainsacchi commented 1 year ago

Hi @felipecampos1994 in a few minutes, v.1.4.1 will be available. Hopefully this should fix your issue -- let me know.

felipecampos1994 commented 1 year ago

Hi @romainsacchi

It did perfectly work now for the {"model":"remind", "pathway":"SSP2-PkBudg500", "year":2030} scenario. I'll try to update and write all scenarios at once next week and will let you know if that also works fine.

Thanks a lot for your help! You guys are doing an amazing job with this project

narierinke commented 1 year ago

Hello,

Sorry for the late reply. It also worked for me.

Best, Narie


De: felipecampos1994 @.> Enviado: quinta-feira, 22 de dezembro de 2022 18:13 Para: polca/premise @.> Cc: narierinke @.>; Mention @.> Assunto: Re: [polca/premise] Error when writing database to Brightway (Issue #93)

Hi @romainsacchihttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fromainsacchi&data=05%7C01%7C%7Cecfbcdf1c55b43f9de3f08dae43fcfda%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638073259963284534%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qXIhJv6zYEYz%2FiQZMAOKoNh02esoycPfgvUpsYG%2F0TA%3D&reserved=0

It did perfectly work now for the {"model":"remind", "pathway":"SSP2-PkBudg500", "year":2030} scenario. I'll try to update and write all scenarios at once next week and will let you know if that also works fine.

Thanks a lot for your help! You guys are doing an amazing job with this project

— Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpolca%2Fpremise%2Fissues%2F93%23issuecomment-1363116199&data=05%7C01%7C%7Cecfbcdf1c55b43f9de3f08dae43fcfda%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638073259963284534%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3DnMG2Ekt2dCH7%2FgJYTR3NXuqRxJHsBP0hCJ2rQGMJc%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAMTB3NBMXJZEZED4E6CIPK3WOSD2VANCNFSM6AAAAAAS5ETZNM&data=05%7C01%7C%7Cecfbcdf1c55b43f9de3f08dae43fcfda%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638073259963284534%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6KwtBmeGbROwTmc8VOGfpZIx9qyNC1K2XYj%2BqlxODxI%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>

moritzos commented 1 month ago

Hi, I have a similar problem to @felipecampos1994 : I created a new scenario database with premise (version 2024.04.23, wurst version 0.4) (source_db: ecoinvent 3.9.1, model: image, pathway: SSP2-RCP19) and then used ‘ndb.update(’electricity")’. Both seem to work. When I then try to export the database to brightway using ‘ndb.write_db_to_brightway()’, I get the following error message: grafik

I thought it might be because I am using python 3.12.2 (according to this example under ‘Requirements’ python 3.10 or higher up to 3.11 is recommended. So I created a new environment with python 3.11.10 (wurst 0.4, premise 2.1.3) and tried again. Then the kernel always crashes when exporting the database to brightway. Kernel

The database seems to have been created afterwards, but it is still unfavourable if the kernel always crashes: grafik

@romainsacchi: Do you have any idea how I can solve this?