force offshore wind from NEP 2023 at correct connection point

[nworbmot]

Projects for offshore wind connection in the North and Baltic Seas are read in and then (in a configurable way) forced into the network with the correction connection point. The default is only to do this for the projects planned until 2030, which mostly already have final investment decisions.

Beware: you may have to adjust the max capacity limits to avoid infeasibilities from forcing in the offshore capacity.

Beware: the connection costs of these forced projects are not calculated yet, or accounted for in the Ariadne database export.

This code was tested with 27 and 49 nodes.

Before asking for a review for this PR make sure to complete the following checklist:

• [ ] Workflow with target rule ariadne_all completes without errors
• [ ] The logic of export_ariadne_variables has been adapted to the changes
• [ ] One or several figures that validate the changes in the PR have been posted as a comment
• [ ] A brief description of the changes has been added to Changelog.md
• [ ] The latest main has been merged into the PR
• [ ] The config has a new prefix of the format YYYYMMDDdescriptive_title

[github-actions[bot]]

Validator Report

I am the Validator. Download all artifacts here. I'll be back and edit this comment for each new commit.

:warning: Config changes detected!

Results may differ due to these changes: ```diff diff --git a/config/config.yaml b/config/config.yaml index 75904f6..ae5e7cb 100644 --- a/config/config.yaml +++ b/config/config.yaml @@ -4,7 +4,7 @@ # docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#run run: - prefix: 20241007_merge_master + prefix: 241008_offshore_nep name: # - CurrentPolicies - KN2045_Bal_v4 @@ -435,8 +435,8 @@ solving: offwind: DE: 2020: 7.8 - 2025: 8 - 2030: 30 + 2025: 11.3 + 2030: 30.4 2035: 70 2040: 70 2045: 70 @@ -635,3 +635,9 @@ transmission_projects: onshore_nep_force: cutin_year: 2020 cutout_year: 2030 + +#beware - may need to increase max offshore +#to avoid infeasibilities +offshore_nep_force: + cutin_year: 2025 + cutout_year: 2030 ```

Ariadne Variables

Comparison

	NRMSE	Pearson
Capacity|Electricity|Storage Converter	0.540	0.016
Final Energy|Residential and Commercial|Liquids|Biomass	0.408	0.000
Price|Primary Energy|Oil	0.350	0.000
Price|Secondary Energy|Gases|Natural Gas	0.349	0.000
Price|Secondary Energy|Gases|Hydrogen	0.345	0.000
Investment|Energy Supply|Electricity|Transmission|Offwind-DC	0.281	0.617
Secondary Energy|Electricity|Hydro	0.281	0.975
Investment|Energy Supply|Electricity|Transmission|Offwind-AC	0.275	0.778
Investment|Energy Supply|Electricity|Wind|Offshore	0.215	0.765
Investment|Energy Supply|Electricity|Transmission	0.201	0.942
Capacity Additions|Electricity|Wind|Offshore	0.201	0.841
Investment|Energy Supply|Electricity|Transmission|DC	0.195	0.955
Investment|Energy Supply|Electricity|Transmission|AC	0.122	0.931
Capacity Additions|Electricity|Solar|PV	0.108	0.969
Capacity Additions|Electricity|Solar	0.108	0.969
Capacity Additions|Electricity|Solar|PV|Rooftop	0.108	0.969
Final Energy|Transportation|Liquids|Biomass	0.104	0.967

NRMSE: Normalized (combined-min-max) Root Mean Square Error Pearson: Pearson correlation coefficient Threshold: NRMSE > 0.1 Only variables reaching the threshold are shown. Find the equivalent plot for all of them below.

Plots

Main branch	Feature branch

General

Plots comparison

Main branch	Feature branch

Files comparison

	Status	NRMSE	MAE (norm)
csvs/market_values.csv	:warning:Changed	0.022	0.05
csvs/curtailment.csv	:warning:Changed	0.000	0.11
csvs/price_statistics.csv	:warning:Changed	0.009	0.24
csvs/cumulative_cost.csv	:warning:Changed	0.001	0.13
csvs/metrics.csv	:warning:Changed	0.001	0.09
csvs/nodal_cfs.csv	:warning: NaN mismatch
csvs/nodal_capacities.csv	:warning: NaN mismatch
csvs/nodal_supply_energy.csv	:warning: NaN mismatch
csvs/nodal_costs.csv	:warning: NaN mismatch
csvs/capacities.csv	:white_check_mark: Almost equal	0.000	0.00
csvs/supply.csv	:white_check_mark: Almost equal	0.002	0.01
csvs/prices.csv	:white_check_mark: Almost equal	0.002	0.00
csvs/cfs.csv	:white_check_mark: Almost equal	0.039	0.00
csvs/costs.csv	:white_check_mark: Almost equal	0.001	0.01
csvs/supply_energy.csv	:white_check_mark: Almost equal	0.000	0.03
csvs/energy.csv	:white_check_mark: Almost equal	0.000	0.02
csvs/weighted_prices.csv	:white_check_mark: Equal

NRMSE: Normalized (combined-min-max) Root Mean Square Error MAE (norm): Mean Absolute Error on normalized data (min-max) Status Threshold: MAE (norm) > 0.05 and NRMSE > 0.3

Model Metrics

Benchmarks

Comparing offshore-nvp (cf8b978) with main (fa953d6). Branch is 15 commits ahead and 0 commits behind. Last updated on 2024-10-09 15:53:26 CEST.

[lindnemi]

Since the built_projects go back to 2009 we get some doubling with the capacities added in add_existing_baseyear. This causes the violation of the capacity limit.

For 2030 the limit has been increased in this PR, but when a different cutin_year is chosen, we get similar problems in 2020 and 2025.

Interestingly, when running with 8 clusters, modify prenetwork does not have to add a new generator. Apparently already add_existing_baseyear places some capacities in land-locked regions. Here are some old networks:

For 8 clusterswe get offshore in Hessen

for 30 cluster we get offshore in northern BW

[lindnemi]

So the cleanest thing would be to add the NEP projects in add_existing_baseyear, or to drop all existing generators in modify_prenetwork and add them back in with correct capacites and locations taken from NEP. Still, this would not yet be sufficient for computing correct investments.

Until the correct way of computing investments is figured out i am reluctant to merge this PR. It will only decrease network expansion.

[lindnemi]

In the previous draft of the PR the capacities from add_existing were corrected for by computing the gap between NEP and existing capacities at a bus. The remaining gap was the extendable capacity. However the NEP and add_existing were not aligned on the bus level and the same capacity could be assigned to different buses. As a consequence the added capacity depended on the number of cluster and was varying between 29 and 32 in 2030 for 27 and 49 clusters respectively. Furthermore, 3 GW of offwind-ac from the NEP were not considered

Now:

• All NEP in Ost and Nordsee are added with their correct capacities. The gap is no longer considered, this is possible because:
  • NEP projects are added in two periods: 2021-2025, 2026-2030. Earlier capacities are taken from add_existing
  • min_capacitiy limits are adjusted to be just above the NEP ausbau.

Open points:

• [ ] The way the land use constrained is hacked seems a bit suspicious. I am worried that the p_nom_max is now too high, because the values are just copied over from the north_sea_duck_node generator. It's no problem as long as the capacity is limited by a global constraint. But should be rewritten later on.
• [ ] Implementation of AC rests on the assumption that the connection point already exists from previous years. might potentially error if later projects connect farther inland. Probably fine
• [ ] Ideally we would have separate, non extendable generators, with "NEP" in their name. Those would have the correct capacities and are not extendable. We could even delete all extendable offwind generators <2030 and only force in capacities in add_existing and in the NEP_force_function. As a caveat, this might throw off n.statistics. Potentiall the new active option in PyPSA might help here.

[lindnemi]

I will merge this without the open points from previous post adressed. However, we should

• [x] add the computation of connection and capacity investments to the exporter

[lindnemi]

The overnight_connection_costs are now directly computed in modify_prenetwork. This way the exporter does not have to be changed. The advantage is that the offshore.csv does not have to be read in the exporter and that endogenous and exogeneous capacities are treated uniformly. The basic idea was to deal with all the complexities arising from the offshoreNEP_force in the same place and not spread these complexities across multiple files. Another approach would have been fine as well though