Integrate Breakthrough Energy Venture data into PyPSA-Earth and run the model

finozzifa commented 3 months ago

Config and Snakefile changes

[ ] Switch off fetch of renewable capacities from IRENA dataset by setting in Config/config.usa_PE.yaml
```
estimate_renewable_capacities:
stats: "False" 
```
[ ] Un-comment clean_osm_data in workflow/pypsa-earth/Snakefile
[ ] Take the plants_merged.csv dataset from PyPSA-US, renamed it to custom_powerplants.csv and place it in workflow/pypsa-earth/data folder

[ ] Take a custom powerplants file. To do it, modify the following key in Config/config.usa_PE.yaml

custom_powerplants: "replace"  #  "false" use only powerplantmatching (ppm) data, "merge" combines ppm and custom powerplants, "replace" use only custom powerplants

[ ] Further modify the Config/config.usa_PE.yaml file such that
- extendable_carriers: Generator: [ ]
- co2limit: 1.4728e+9 and co2base: 226.86e+6
- load_shedding: false

Files plants_merged.csv custom_powerplants.csv

Model execution steps

[x] Check what cutout you are using, whether the one week cutout or the one provided at northern emisphere
[x] build the powerplant file with snakemake -call build_powerplants --cores all --printshellcmds --configfile ../../Config/config.usa_PE.yaml
[x] run the model with snakemake -call solve_all_networks --cores all --printshellcmds --configfile ../../Config/config.usa_PE.yaml
[x] using the notebook modify_network.ipynb modify the network pypsa-earth/networks/US_2021/elec_s_10_ec_lcopt_Co2L-24H.nc so that
```
n.generators.loc[:,  "p_nom_extendable"] = False
```
[x] run the model with snakemake -call solve_all_networks --cores all --printshellcmds --configfile ../../Config/config.usa_PE.yaml

Files data_validation.zip

SermishaNarayana commented 3 months ago

The data integration is a 2-step process as follows

[x] Integration of installed capacity data from BEV
[x] Integration of demand profile from BEV

finozzifa commented 3 months ago

Installed capacity

Modify the custom_powerplants.csv:

[x] BEV data does not distinguish between Hydro and PHS.

Modify demand profile

Note: the data/ssp2-2.6/2030/era5_2013/NorthAmerica.csv is most likely downloaded in rule retrieve_databundle_light. So we should:

To modify demand_profile

[x] modify `geothermal-modelling/data/NorthAmerica.csv' including the demand profile from BEV
[x] copy it to data/ssp2-2.6/2030/era5_2013/NorthAmerica.csv
[x] execute snakemake -call build_demand_profiles --cores all --printshellcmds --configfile config.yaml

This part happens between the rule build_powerplant and rule add_electricity.

[x] execute snakemake -call add_electricity --cores all --printshellcmds --configfile config.yaml

SermishaNarayana commented 3 months ago

Notebook to analyse the model outputs

Analysis_29_07.zip

finozzifa commented 3 months ago

Notebook to modify NorthAmerica.csv (updated on August 1st 2024 at 11:46 CET)

modify_demand.ipynb.zip

danielelerede-oet commented 3 months ago

The difference between conventional (dam and ROR) and PHS in hydroelectric capacity (not present in the BEV dataset) is now envisaged in custom_powerplants.

1) Check on the powerplantmatching dataset (correspondence between ppm and BEV found for just 5 plants and several mistakes spotted - e.g. coal/gas power plants mistaken for hydro power plants) 2) Classification of hydro power plants and double check performed based on: https://clui.org/projects/offstream/pumped-storage-facilities-usa (specifically for PHS) https://en.wikipedia.org/wiki/List_of_run-of-the-river_hydroelectric_power_stations (specifically for ROR plants) https://www.certrec.com/blog/certrec-market-research/top-ten-largest-hydro-plants-in-the-united-states/ https://hydroreform.org/on-your-river/ 3) Final check on the validation of installed capacity (PyPSA-Earth VS EIA)

installed_capacity

finozzifa commented 3 months ago

Including the demand profile from BEV into PyPSA-Earth

Progress was made in including the BEV demand profile into PyPSA-Earth. The steps were:

download the BEV reference data from this USATestSystem.zip. In particular we used the data set USATestSystem/base_grid/demand.csv. The csv contains a time series for 2016 for a few hundred buses
2016 was a leap year. I have therefore excluded the February 29th 2016 data
sum the contribution of each bus for each day
in the NorthAmerica.csv dataset, I replaced the rows corresponding to the US with the BEV data described above
I executed the rule build_demand_profile and obtained the pypsa-earth/resources/US_2021/demand_profiles.csv
as an extra sanity check, I compare the data from pypsa-earth/resources/US_2021/demand_profiles.csv with the data from USATestSystem/base_grid/demand.csv (aggregated over the buses) and got a match

Solve the network with the updated installed capacity and demand

This point is still in progress

danielelerede-oet commented 3 months ago

Comparison of electricity generation results after the updates of costs.csv:

Corresponding to the merit order below:

open-energy-transition / geothermal-modelling