Add spec to check whether all primary demand is accounted for

[michieldenhaan]

The sum of the demands of all primary energy nodes (primary_energy_demand group) should equal the sum of the primary demand of all final demand nodes and all export nodes. In other words, all primary energy demand in the graph is either used domestically (final demand) or exported. Similarly, the CO2 associated with the primary energy nodes should equal the CO2 associated with the final demand and export nodes.

This ensures that no demand or emissions are unaccounted for.*

We should check this with MT specs. I.e. SUM(V(G(primary_energy_demand), primary_demand)) should equal SUM(V(G(final_demand_group), primary_demand)) + SUM(V(G(energy_export), primary_demand)) and SUM(V(G(primary_energy_demand), primary_co2_emission)) should equal SUM(V(G(final_demand_group), primary_co2_emission)) + SUM(V(G(energy_export), primary_co2_emission))

Currently this test will fail for:

• ~Network gas due to unused heat in industry not being in the final demand group~
• ~Network gas and hydrogen due to an issue with burners with efficiencies above 100%~
• Crude oil due to refinery gas not being in the final demand group

* We could introduce a third category for nodes which are neither final demand nor export, but that's another discussion

Update: I guess we have to include curtailment as well, as curtailed electricity is neither export nor final_demand.

[michieldenhaan]

This spec fails for scenarios with excess electricity. For example, in the II3050 EU scenario 188 PJ of primary demand in the graph cannot be traced to final demand, export or curtailment.

This has two causes:

1. Some uses of excess electricity never end up as final demand or export. E.g. industry_chemicals_other_flexibility_p2h_hydrogen_electricity uses electricity to produce heat, but neither this electricity nor the heat passes through a 'final demand' node. This is discussed in an earlier issue and should be relatively easy to fix: https://github.com/quintel/etmodel/issues/2527

2. A more fundamental problem is that excess electricity is taken from the grid nodes in the graph by Merit and transferred to the energy_production_excess_electricity node. This node has no incoming connections to any other node and is in the primary_energy_demand group. This has two side effects:

   • We can no longer trace the primary carrier that produced the excess electricity. E.g. Suppose all electricity is produced with a must-run biogas CHP. A node using excess electricity will then have a primary demand of 'electricity', not biogas because the energy is transferred to energy_production_excess_electricity and we have no way of tracing where it came from.
   • We can no longer account for conversion losses. Again suppose all electricity is produced with a must-run biogas CHP. The primary demand of a node using excess electricity will equal the amount of electricity that went into the node. This neglects the conversion losses that occured to produce this excess electricity. The effect of this can be substantial. We see this in the EU scenario: 188 PJ of primary demand is missing, even though only 135 PJ of excess electricity is produced.

This wouldn't be too much of a problem if all excess electricity came from solar, wind etc (which isn't necessarily the case btw) because their primary carrier is electricity (no side effect 1) and they don't have conversion losses (no side effect 2). However, theprimary_demand and primary_co2_emission methods operate on the annual graph, so the 188 PJ missing primary demand corresponds to the annual electricity mix. This could include coal, natural gas etc. We discussed this earlier here: https://github.com/quintel/etsource/issues/2172.

@marliekeverweij @ChaelKruip @antw Is this something to look into in the upcoming flex project? One simple option might be to connect all flex technologies to a final demand node or lv/mv/hv grid node, rather than to energy_production_excess_electricity. We could then get rid of this node and Merit no longer needs to transfer energy to it. Instead, all electricity demand ('flex' demand and regular demand) is treated the same in the graph, i.e. ultimately traceable to the nodes where the electricity came from. (This also aligns well with the changes we're making to flex, allowing flex option to also use power from dispatchable power plants if this is economically feasible, thereby fading the strict distinction between 'excess' electricity and 'regular' electricity)

Screenshots from EU scenario:

110 PJ is taken from energy_power_hv_network_electricity:

and transfered to energy_production_excess_electricity:

[antw]

One simple option might be to connect all flex technologies to a final demand node or lv/mv/hv grid node, rather than to energy_production_excess_electricity. We could then get rid of this node and Merit no longer needs to transfer energy to it.

It makes intuitive sense for flexibility technologies to be connected correctly to the graph so that flows can be fully accounted for. I don't have an opinion for where they should be connected.

One small point of clarification: Merit doesn't explicitly transfer energy to the excess_electricity node. Each flex technology is assigned a demand based on the outcome of the Merit calculation, and ETEngine then calculates the value for the excess_electricity node.