Units issue in flows_biosphere_39.csv file

[tpsarblaster]

I encountered an error when executing write_superstructure_db_to_brightway() (console log included below). This is when using today's release of premise 1.8.0 (this commit), installed in a new environment and making sure that the biosphere3 database is updated (bw2io 0.8.10, with ecoinvent 3.9.1).

Investigating the file .../premise/data/utils/export/flows_biosphere_39.csv used to create bio_dict, there is indeed no flow matching ('Gas, natural', 'natural resource', 'in ground', 'Sm3'), since this flow is in the .csv file with the unit standard cubic meter instead of Sm3. Changing the flows in the .csv file which have the standard cubic meter unit to Sm3 appears to fix the issue.

Building superstructure database...
Traceback (most recent call last):

  Cell In[6], line 1
    ndb.write_superstructure_db_to_brightway(name="test REMIND SSP2-Base")

  File ~\anaconda3\envs\premise18\lib\site-packages\premise\ecoinvent_modification.py:1390 in write_superstructure_db_to_brightway
    self.database = generate_superstructure_db(

  File ~\anaconda3\envs\premise18\lib\site-packages\premise\export.py:920 in generate_superstructure_db
    df, new_db, _ = generate_scenario_difference_file(

  File ~\anaconda3\envs\premise18\lib\site-packages\premise\export.py:760 in generate_scenario_difference_file
    new_db = pool.map(

  File ~\anaconda3\envs\premise18\lib\multiprocessing\pool.py:367 in map
    return self._map_async(func, iterable, mapstar, chunksize).get()

  File ~\anaconda3\envs\premise18\lib\multiprocessing\pool.py:774 in get
    raise self._value

  File ~\anaconda3\envs\premise18\lib\multiprocessing\pool.py:125 in worker
    result = (True, func(*args, **kwds))

  File ~\anaconda3\envs\premise18\lib\multiprocessing\pool.py:48 in mapstar
    return list(map(*args))

  File ~\anaconda3\envs\premise18\lib\site-packages\premise\export.py:668 in generate_new_activities
    act["exchanges"].extend(

  File ~\anaconda3\envs\premise18\lib\site-packages\premise\export.py:669 in <genexpr>
    get_exchange(i, acts_ind, db_name, version, amount=m[i, k, 0]) for i in v

  File ~\anaconda3\envs\premise18\lib\site-packages\premise\export.py:439 in get_exchange
    "input": ("biosphere3", correct_biosphere_flow(name, cat, unit, version))

  File ~\anaconda3\envs\premise18\lib\site-packages\premise\export.py:425 in correct_biosphere_flow
    return bio_dict[(name, main_cat, sub_cat, unit)]

KeyError: ('Gas, natural', 'natural resource', 'in ground', 'Sm3')

[romainsacchi]

Hi @tpsarblaster , yes, this is probably because your biosphere database was generated by an older version of bw2io (e.g., 0.8.8). Not a premise issue as far as I can tell, since the unit proposed is correct for bw2io 0.8.10 -- but I agree that it is annoying.

[romainsacchi]

@tpsarblaster do you confirm that your biosphere3 database was generate by bw2io 0.8.10?

[romainsacchi]

As a temporary fix, I added the UUID for the flow with the Sm3 unit.

[tpsarblaster]

@romainsacchi Yes. Before even installing premise to the environment I was using in this case, the biosphere3 database had been updated using bw2io 0.8.10.

[romainsacchi]

@tpsarblaster OK, that would mean bw2io converts some units when creating a new biosphere3 database but not when updating an existing one.

This is the list of unit conversions operated by bw2io 0.8.10 upon database creation: https://github.com/brightway-lca/brightway2-io/blob/main/bw2io/units.py

In any case, it should be fixed now in the next release, thanks.