recursive_factor_without_losses method might be employed in calculations for which it shouldn't be employed

[RobTerwel]

Background

This is a follow-up to https://github.com/quintel/etengine/issues/860#issuecomment-241760695 .

There are two recursive_factor methods: recursive_factor and recursive_factor_without_losses.

These methods traverse the graph (left-to-right, child-to-parent) starting from the node for which it is called, following all branches until it has found all end points. The difference between these methods is that the former takes into account losses, while the latter doesn't, in determining the 'weight' of links. As the internal description rightly stresses, the recursive_factor_without_losses should not be used for attributes related to the demand of converters - as losses entail total primary demand is greater than final demand and that has an effect on the weight of outcomes.

Possible problem for some attributes

Currently, this recursive_factor_without_losses method is used to calculate the following attributes:

• weighted_carrier_cost_per_mj
• weighted_carrier_co2_per_mj
• sustainability_share

I think it should be good to investigate whether these three attributes can be specified without taking into account losses that occurred along the way.

Some thoughts

At first hand they do all seem to be related to demand to me.

Carrier cost and co2 per mj intuitively seem to depend on demand, as high losses should lead to higher a cost per mj and higher co2 emissions per mj (for non-sustainable carriers) of the carriers entering the node for which the method is used.

As to the sustainability share, I think choosing between these two recursive factors boils down to choosing to determine the sustainablility_share between FD carriers or PD carriers.

Including @ChaelKruip @jorisberkhout @AlexanderWirtz . Re-assign at will, and feel free to expand as the above serves as a short upshot of discussions @ChaelKruip and I had.

Carrier cost and co2 per mj intuitively seem to depend on demand, as high losses should lead to higher a cost per mj and higher co2 emissions per mj (for non-sustainable carriers) of the carriers entering the node for which the method is used.

Thinking about this some more I am not convinced anymore that weighted_carrier_co2_per_mj should be treated on equal footing with weighted_carrier_cost_per_mj. The latter needs to take losses into account because balsamic vinegar becomes more expensive as its watery part vaporises. For the CO2-content of a carrier, however, it depends on what you mean with content:

• Definition 1: the CO2 present in the primary carriers which have led to this carrier
• Definition 2: the CO2 which would be released by energetic use (burning) of this fuel

For the first definition to work, the losses need to be taken into account and the recursive_factor_without_losses method is inappropriate IMO.

For the second definition to work, I think the recursive_factor_without_losses method could work fine as long as it does the following (I think this is what it does):

• Follow each incoming edge back to its parents until for each recursive path the carrier_co2_per_mj is known from a carrier.ad file.
• Weigh the carrier_co2_per_mj's of the paths with the parent shares (the ones on the right of the nodes).

@antw can you confirm/check this please?

NOTE: The weighted_carrier_co2_per_mj method is only used in one query as far as I can detect.

[antw]

• Follow each incoming edge back to its parents until for each recursive path the carrier_co2_per_mj is known from a carrier.ad file.
• Weigh the carrier_co2_per_mj's of the paths with the parent shares (the ones on the right of the nodes). @antw can you confirm/check this please?

Hmm, no. This is the case for weighted carrier cost per MJ, but apparently not for weighted carrier CO₂ per MJ. The former will stop once it encounters a carrier with a cost_per_mj (or if it hits electricity/steam_hot_water), but the latter only stops when it hits the far-right of the graph (there are no more parents through which to traverse).

They both used to continue to the far-right of the graph, but cost was changed in #708 to fix #707.

[antw]

To add:

• if CO₂ per MJ should stop when it encounters a carrier with a co2_per_mj defined (like cost does)... this is a trivial change.
• I don't know why the cost traversal stops when encountering electricity or steam/hot-water. The comment says: "because electricity and steam_hot_water are calculated seperately these are excluded from this calculation", but doesn't elaborate further. Unfortunately that code predates both my time at QI and ETEngine itself, so I don't know what the original motivation was. 😬

[ChaelKruip]

I don't know why the cost traversal stops when encountering electricity or steam/hot-water. The comment says: "because electricity and steam_hot_water are calculated seperately these are excluded from this calculation", but doesn't elaborate further.

Perhaps because the cost of both heat and electricity is more complex than than just mixing the costs of carriers used to make it? For electricity we seem to use the interpretation that the cost of electricity is related to all costs of the producing technologies as can be seen in this gquery.

Interestingly, the weighted_carrier_cost_per_mj method returns zero for all final demand converters except the energy_cokesoven_consumption_coal_gas in the scenario that I tried. I don't understand how the method can return non-zero values for electricity 😟

[antw]

Interestingly, the weighted_carrier_cost_per_mj method returns zero for all final demand converters except the energy_cokesoven_consumption_coal_gas in the scenario that I tried.

If you were using TXT_TABLE, know that it rounds numbers; I don't know why I thought that was a good idea.

You can hover your mouse over a number to see its real value, or use one of the other views (Text, CSV, or TSV):

TXT_TABLE(G(final_demand_group), key, weighted_carrier_cost_per_mj)

+------------------------------------------------------------------------+------------------------------+
|                                  key                                   | weighted_carrier_cost_per_mj |
+------------------------------------------------------------------------+------------------------------+
| buildings_final_demand_solar_thermal                                   | 0.0                          |
| buildings_final_demand_steam_hot_water                                 | 0.0                          |
| buildings_final_demand_coal                                            | 0.003132054822806707         |
| buildings_final_demand_electricity                                     | 0.0                          |
| buildings_final_demand_network_gas                                     | 0.008640576736594994         |
| buildings_final_demand_crude_oil                                       | 0.017318684168331922         |
| buildings_final_demand_wood_pellets                                    | 0.007881766666666665         |
| households_final_demand_solar_thermal                                  | 0.0                          |
| households_final_demand_wood_pellets                                   | 0.007881766666666665         |
| households_final_demand_network_gas                                    | 0.008640576736594994         |
| households_final_demand_crude_oil                                      | 0.017318684168331922         |
| households_final_demand_steam_hot_water                                | 0.0                          |
| households_final_demand_electricity                                    | 0.0                          |
| households_final_demand_coal                                           | 0.003132054822806707         |
| industry_own_use_coal                                                  | 0.003132054822806707         |
| industry_final_demand_coal_gas                                         | 0.00252209440770556          |
| industry_final_demand_network_gas_non_energetic                        | 0.008640576736594994         |
| industry_final_demand_for_chemical_other_crude_oil_non_energetic       | 0.017318684168331922         |
| industry_final_demand_for_chemical_fertilizers_crude_oil_non_energetic | 0.017318684168331922         |
| industry_final_demand_crude_oil_non_energetic                          | 0.017318684168331922         |
| industry_final_demand_coal_non_energetic                               | 0.003132054822806707         |
| industry_final_demand_for_other_ict_electricity                        | 0.0                          |
| industry_final_demand_wood_pellets_non_energetic                       | 0.007881766666666665         |
| industry_final_demand_coal                                             | 0.003132054822806707         |
| industry_final_demand_wood_pellets                                     | 0.007881766666666665         |
| industry_final_demand_network_gas                                      | 0.008640576736594994         |
| industry_final_demand_electricity                                      | 0.0                          |
| industry_final_demand_steam_hot_water                                  | 0.0                          |
| industry_final_demand_crude_oil                                        | 0.017318684168331922         |
| other_final_demand_network_gas                                         | 0.008640576736594994         |
| other_final_demand_wood_pellets                                        | 0.007881766666666665         |
| other_final_demand_coal                                                | 0.003132054822806707         |
| other_final_demand_electricity                                         | 0.0                          |
| other_final_demand_steam_hot_water                                     | 0.0                          |
| other_final_demand_crude_oil                                           | 0.017318684168331922         |
| other_final_demand_crude_oil_non_energetic                             | 0.017318684168331922         |
| energy_power_sector_own_use_electricity                                | 0.0                          |
| energy_cokesoven_consumption_coal_gas                                  | 0.002420928247145298         |
| energy_export_coal_gas                                                 | 0.00252209440770556          |
| transport_final_demand_lpg                                             | 0.041471884699634934         |
| transport_final_demand_hydrogen                                        | 0.00473730005934254          |
| transport_final_demand_compressed_network_gas                          | 0.00804437694176994          |
| transport_final_demand_electricity                                     | 0.0                          |
| transport_final_demand_crude_oil_non_energetic                         | 0.017318684168331922         |
| transport_final_demand_kerosene                                        | 0.00932795226218376          |
| transport_final_demand_heavy_fuel_oil                                  | 0.00870296296296296          |
| transport_final_demand_diesel                                          | 0.03540634047608419          |
| transport_final_demand_bio_lng                                         | 0.017                        |
| transport_final_demand_lng                                             | 0.013081205430645913         |
| transport_final_demand_bio_ethanol                                     | 0.0393325353449389           |
| transport_final_demand_gasoline                                        | 0.042703492692021006         |
| transport_final_demand_biodiesel                                       | 0.0237596416236569           |
| transport_final_demand_coal                                            | 0.003132054822806707         |
| agriculture_final_demand_crude_oil                                     | 0.017318684168331922         |
| agriculture_final_demand_wood_pellets                                  | 0.007881766666666665         |
| agriculture_final_demand_steam_hot_water                               | 0.0                          |
| agriculture_final_demand_electricity                                   | 0.0                          |
| energy_heat_network_unused_steam_hot_water                             | 0.0                          |
| agriculture_final_demand_network_gas                                   | 0.008640576736594994         |
+------------------------------------------------------------------------+------------------------------+

[ChaelKruip]

If you were using TXT_TABLE, know that it rounds numbers; I don't know why I thought that was a good idea.

I was 😆 ... thanks

[ChaelKruip]

if CO2 per MJ should stop when it encounters a carrier with a co2_per_mj defined (like cost does)... this is a trivial change.

I have tried to do this:

  def weighted_carrier_co2_per_mj
    fetch(:weighted_carrier_co2_per_mj) do
      recursive_factor_without_losses(:weighted_carrier_co2_per_mj_factor)
    end
  end

  def weighted_carrier_co2_per_mj_factor(link)
    # because electricity and steam_hot_water have no intrinsic co2
    # these are excluded from this calculation
    if link
      if (link.carrier.electricity? || link.carrier.steam_hot_water?)
        0.0
      elsif link.carrier.co2_conversion_per_mj || right_dead_end?
        link.carrier.co2_conversion_per_mj
      else
        nil # continue to the right
      end
    else
      nil # continue to the right
    end
  end

And it seems to work for some carriers. For mixed carriers such as network_gas I'm running into the following situation:

At the final demand level (households in this case), the weighted_carrier_co2_per_mj is (erroneously) zero: but the next node to the right does have a non-zero weighted_carrier_co2_per_mj:

Any ideas @antw why this might be happening?

[antw]

Network gas has a co2_conversion_per_mj defined.

carriers/network_gas.ad: - co2_conversion_per_mj = 0.0

[ChaelKruip]

Network gas has a co2_conversion_per_mj defined.

What nonsense! I'll remove that attribute after some consideration 😄

[ChaelKruip]

Closing this as I think that two of the three methods are currently (after https://github.com/quintel/etengine/commit/7e805d85b167b77a36faa727c6211e33b9e8c11d) working properly.

Making a new issue for sustainability_share