Where are the product systems?

[dt-woods]

The electricity baseline (as of 2016) includes 68 product systems comprised of consumption mixes at the user level for 57 balancing authorities, 10 FERC regions, and the US.

These consumption mixes at user level are created in eia_trans_dist_grid_loss.py module and should be linked to their at-grid consumption mix processes, which, themselves, should be linked to the appropriate at-grid generation processes. See below for my interpretation of how this looks in the electricity baseline.

The main method within main.py ends at the writing of the distribution mix processes (i.e., the at-grid consumption mixes) and there is no product system creation.

Main questions are:

1. Where does product system creation take place?

---

Overview

The electricity baseline provides two sides to the electricity story: electricity generation (i.e., creating the electricity from a fuel technology, such as wind or coal) and electricity distribution (i.e., delivering electricity to an end user, through transmission and distribution networks).

The processes provided in the electricity baseline mirror this at three levels:

• Electricity; at grid; generation mix
  • The LCA inventory behind this process is based on the power plants operating within the balancing authority. These power plants may produce electricity using a variety of fuel technologies, such as coal, oil, gas, nuclear, wind, solar, and hydro.
• Electricity; at grid; consumption mix
  • The electricity distributed is not solely the electricity generated by a given balancing authority. Electricity is a commodity traded between balancing authorities; therefore, to get a holistic view of the inventories (and thus the impacts) that go into the electricity consumed by an end user, the electricity mix of all participating balancing authorities is provided in the "at grid" consumption mix.
  • The "at grid" consumption mix links to one or more "at grid" generation mixes based on balancing authority electricity trading data.
• Electricity; at user; consumption mix
  • Electricity consumed is not the full picture of electricity generated due to losses incurred in the distribution and transmission networks; therefore, electricity "at user" is an adjusted quantity of electricity "at grid" based on a national average transmission and distribution loss factor.

[m-jamieson]

That has to happen in openLCA itself. I use an openLCA IPC script for that. I haven't run this in a while, and it would've been using an older version of IPC but should give you an idea.

Additionally

import olca
import regex as re
from datetime import datetime
import pytz

#The port is set within openLCA when you start IPC
#The default port (8080) is likely used by something else.
olca_client=olca.Client(port=60000)
matching_regex=[
    "Electricity - .+ - .+", #Technology processes (COAL plants)
    "Electricity; at user; consumption mix - .+ - BA", #BA consumption mixes
    "Electricity; at user; consumption mix - .+ - FERC", #FERC consumption mixes
    "Electricity; at grid; generation mix - .+", #BA generation mixes
]
combined = "(" + ")|(".join(matching_regex) + ")"
systems = olca_client.get_all(model_type=olca.Process) #I think to speed this up can use something like .get_descriptor
matching_processes = [x.name for x in systems if re.match(combined,x.name)]
for process in matching_processes:     
    olca_process = olca_client.find(model_type=olca.Process,name=process)
    new_system_ref = olca_client.create_product_system(
        process_id=olca_process.id,
        default_providers="only",
        preferred_type="SYSTEM_PROCESS",
    ) #This returns a reference, not an olca.ProductSystem
    new_system=olca_client.get(olca.ProductSystem,new_system_ref.id)
    new_system.olca_type = olca.schema.ProductSystem.__name__

    new_system.description = (
        f"Generated using OLCA-IPC."
    )
    #name the product system the same as your process
    new_system.name = olca_process.name
    model_type_str = f"{olca.ModelType.PRODUCT_SYSTEM}"
    new_system.version = "1.0.0"
    new_system.olca_type = olca.schema.ProductSystem.__name__
    new_system.category_path = []
    datetime_str = datetime.now(pytz.utc).isoformat()
    new_system.last_change=datetime_str
    olca_client.update(new_system)

I also executed some SQL in openLCA to clean some things up for posting to the federal LCA commons:

#deleting flows that aren't used in any processes to reduce the size of the database.
delete
from tbl_flows
where not exists
(select f_flow from 
(select distinct f_flow
from tbl_exchanges) b2
where tbl_flows.id=b2.f_flow);

#deleting impact factors that link to now deleted flows.
delete
from tbl_impact_factors
where not exists
(select id from tbl_flows
where tbl_impact_factors.f_flow=tbl_flows.id);

#updating the product system description to provide a bit more metadata.
update
tbl_product_systems
set tbl_product_systems.description = 'This product system was created in openLCA by linking default providers. The processes were generated by ElectricityLCI (https://github.com/USEPA/ElectricityLCI) version 1.0.1 using the ELCI_1 configuration.   First created: 2020-08-03T14:51:32  Linking approach during creation: Only link default providers; Preferred process type: System process'
where tbl_product_systems.name like 'Electricity; at user; consumption mix - %';

#Setting the product system version to align with the version of eLCI.
update
tbl_product_systems
set tbl_product_systems.version = 4294967297;
update
tbl_processes
set tbl_processes.version = 4294967297;

[dt-woods]

Got it. Thanks, Matt. Looks outdated by the first import statement, but it's not my first time updating olca calls to version 2 😜.

Our new NetlOlca class can handle both JSON-LD zip file and openLCA IPC projects as they provide essentially the same functionality, just different syntax. I don't see anything inhibiting this to be done in electricityLCI.

[dt-woods]

I am a little worried about the two-hour runtime, though.

[m-jamieson]

2 hours just to generate the product systems or 2 hours to run the anlaysis portion in the notebook I sent? 2 hours does seem excessive. I recall maybe 20-30 seconds per product system just to create and save them. And I should say the bulk of this time is waiting for openLCA to do it's thing.

[dt-woods]

I was responding to the comment in the Jupyter notebook you sent, Matt. Haven't tested these methods out yet, but am working on their implementation, along with #217.

[dt-woods]

I thought I could be clever and implement the Product System without the IPC server, but it's more fuss than fun. Therefore, we will endeavor to use olca-ipc's create_product_system in openLCA. I will move this task over the NetlOlca Python package as a new use case, as that is the tool we're using for interfacing with openLCA.

[dt-woods]

That has to happen in openLCA itself. I use an openLCA IPC script for that. I haven't run this in a while, and it would've been using an older version of IPC but should give you an idea.

Additionally

import olca
import regex as re
from datetime import datetime
import pytz

#The port is set within openLCA when you start IPC
#The default port (8080) is likely used by something else.
olca_client=olca.Client(port=60000)
matching_regex=[
    "Electricity - .+ - .+", #Technology processes (COAL plants)
    "Electricity; at user; consumption mix - .+ - BA", #BA consumption mixes
    "Electricity; at user; consumption mix - .+ - FERC", #FERC consumption mixes
    "Electricity; at grid; generation mix - .+", #BA generation mixes
]
combined = "(" + ")|(".join(matching_regex) + ")"
systems = olca_client.get_all(model_type=olca.Process) #I think to speed this up can use something like .get_descriptor
matching_processes = [x.name for x in systems if re.match(combined,x.name)]
for process in matching_processes:     
    olca_process = olca_client.find(model_type=olca.Process,name=process)
    new_system_ref = olca_client.create_product_system(
        process_id=olca_process.id,
        default_providers="only",
        preferred_type="SYSTEM_PROCESS",  # <-- HERE
    ) #This returns a reference, not an olca.ProductSystem
    new_system=olca_client.get(olca.ProductSystem,new_system_ref.id)
    new_system.olca_type = olca.schema.ProductSystem.__name__

    new_system.description = (
        f"Generated using OLCA-IPC."
    )
    #name the product system the same as your process
    new_system.name = olca_process.name
    model_type_str = f"{olca.ModelType.PRODUCT_SYSTEM}"
    new_system.version = "1.0.0"
    new_system.olca_type = olca.schema.ProductSystem.__name__
    new_system.category_path = []
    datetime_str = datetime.now(pytz.utc).isoformat()
    new_system.last_change=datetime_str
    olca_client.update(new_system)

@m-jamieson, in the example code you provided above, when creating the product systems you select preferred type to be SYSTEM_PROCESS. I'm finding that all processes in the baseline are of type UNIT_PROCESS, and I'm failing the linking. Should the baseline processes be LCI processes or unit processes? I thought system processes were roll-ups, so I may be missing something here.

[m-jamieson]

Things like coal and natural gas rollups should be of type system_process because they are rollups of other systems. That being said, it's only preferred. I don't think we should run into any collisions where the name of a sytem_process = name of a unit_process. Maybe something changed with olca 2.0 though that makes this only use system_process?

[dt-woods]

Okay. I'll keep testing.

[dt-woods]

I wasn't able to generate the linkages for a product system using the IPC. I posed the question to GreenDelta; see https://github.com/GreenDelta/olca-ipc.py/issues/31. We'll see if they provide us with a better solution; if not, the likeliest option is to generate the product systems programmatically, then manually go through them and generate the complete supply chains in openLCA.

[dt-woods]

Back to the process of automating product systems with ElectricityLCI. The problem with building the supply chain was recently solved in the NetlOlca class, which is now being copied over into olca_jsonld_writer.py.