search

Breakthrough-Energy / PowerSimData

Simulation framework
https://breakthrough-energy.github.io/docs/
MIT License
52 stars 40 forks source link

Change load zone definition #714

Closed rouille closed 1 year ago

rouille commented 1 year ago

Pull Request doc

Purpose

Use load buses instead of buses as load zone

What the code is doing

N/A

Testing

Existing unit tests

Where to look

Load buses are used for geographical mappings. The number of load buses now matches the number of nodes of the network as given by the reduction parameter (see below)

Usage Example/Visuals

>>> from powersimdata import Grid
>>> entso = Grid("Europe", source="europe_tub", reduction=128)
Title: PyPSA-Eur: An Open Optimisation Model of the European Transmission System (Dataset)
Publication date: 2022-09-20
Version: v0.6.1
DOI: 10.5281/zenodo.7251657
networks.zip has been downloaded previously
WARNING:pypsa.io:Importing network from PyPSA version v0.20.0 while current version is v0.21.2. Read the release notes at https://pypsa.readthedocs.io/en/latest/release_notes.html to prepare your network for import.
INFO:pypsa.io:Imported network elec_s_128_ec.nc has buses, carriers, generators, lines, links, loads, storage_units, stores
>>> len(entso.model_immutables.zones["id2abv"])
128
>>> entso.model_immutables.zones["id2abv"]
{0: 'AL', 1: 'AT', 2: 'AT', 3: 'BA', 4: 'BE', 5: 'BE', 6: 'BE', 7: 'BG', 8: 'CH', 9: 'CZ', 10: 'CZ', 11: 'DE', 12: 'DE', 13: 'DE', 14: 'DE', 15: 'DE', 16: 'DE', 17: 'DE', 18: 'DE', 19: 'DE', 20: 'DE', 21: 'DE', 22: 'DE', 23: 'DE', 24: 'DE', 25: 'DE', 26: 'DE', 27: 'DE', 28: 'DE', 29: 'DE', 30: 'DE', 31: 'DK', 32: 'DK', 33: 'EE', 34: 'ES', 35: 'ES', 36: 'ES', 37: 'ES', 38: 'ES', 39: 'ES', 40: 'ES', 41: 'ES', 42: 'ES', 43: 'ES', 44: 'FI', 45: 'FI', 46: 'FI', 47: 'FR', 48: 'FR', 49: 'FR', 50: 'FR', 51: 'FR', 52: 'FR', 53: 'FR', 54: 'FR', 55: 'FR', 56: 'FR', 57: 'FR', 58: 'FR', 59: 'FR', 60: 'FR', 61: 'FR', 62: 'FR', 63: 'FR', 64: 'FR', 65: 'FR', 66: 'GB', 67: 'GB', 68: 'GB', 69: 'GB', 70: 'GB', 71: 'GB', 72: 'GB', 73: 'GB', 74: 'GB', 75: 'GB', 76: 'GB', 77: 'GB', 78: 'GB', 79: 'GB', 80: 'GR', 81: 'HR', 82: 'HU', 83: 'IE', 84: 'IT', 85: 'IT', 86: 'IT', 87: 'IT', 88: 'IT', 89: 'IT', 90: 'IT', 91: 'IT', 92: 'IT', 93: 'IT', 94: 'IT', 95: 'IT', 96: 'IT', 97: 'LT', 98: 'LU', 99: 'LV', 100: 'ME', 101: 'MK', 102: 'NL', 103: 'NL', 104: 'NL', 105: 'NL', 106: 'NO', 107: 'NO', 108: 'NO', 109: 'NO', 110: 'NO', 111: 'PL', 112: 'PL', 113: 'PL', 114: 'PL', 115: 'PL', 116: 'PT', 117: 'PT', 118: 'RO', 119: 'RO', 120: 'RS', 121: 'SE', 122: 'SE', 123: 'SE', 124: 'SE', 125: 'SE', 126: 'SI', 127: 'SK'}
>>>

Time estimate

2min

rouille commented 1 year ago

@FabianHofmann, can you have a look at this. Thanks!

rouille commented 1 year ago

I believe, this should be ready to be merged.

BainanXia commented 1 year ago

As discussed with @jenhagg, one potential issue is we only have load zone tag for load buses but not for all buses. I believe it should be fine when running base grid without change table. We might need to revisit this to give all buses load zone tags to make full change table schema work.

rouille commented 1 year ago

My understanding is that the substations bear the same name and coordinates than the load buses and and all buses are connected to a substation. Therefore you get the mapping with:

>>> from powersimdata import Grid
>>> entso = Grid("Europe", source="europe_tub", reduction=128)
Title: PyPSA-Eur: An Open Optimisation Model of the European Transmission System (Dataset)
Publication date: 2022-09-20
Version: v0.6.1
DOI: 10.5281/zenodo.7251657
networks.zip has been downloaded previously
WARNING:pypsa.io:Importing network from PyPSA version v0.20.0 while current version is v0.21.2. Read the release notes at https://pypsa.readthedocs.io/en/latest/release_notes.html to prepare your network for import.
INFO:pypsa.io:Imported network elec_s_128_ec.nc has buses, carriers, generators, lines, links, loads, storage_units, stores
>>> entso.sub
        name  interconnect_sub_id        lat        lon  interconnect    pypsa_y    pypsa_x
sub_id                                                                                     
AL1 0    NaN                  NaN  41.587568  19.897914           NaN  41.587568  19.897914
AT1 0    NaN                  NaN  47.354096  11.237535           NaN  47.354096  11.237535
AT1 1    NaN                  NaN  47.671625  15.083487           NaN  47.671625  15.083487
BA1 0    NaN                  NaN  43.811367  18.122704           NaN  43.811367  18.122704
BE1 0    NaN                  NaN  51.005779   4.451911           NaN  51.005779   4.451911
...      ...                  ...        ...        ...           ...        ...        ...
SE2 2    NaN                  NaN  63.114629  16.174075           NaN  63.114629  16.174075
SE2 3    NaN                  NaN  67.551535  18.179626           NaN  67.551535  18.179626
SE2 4    NaN                  NaN  59.851523  16.888635           NaN  59.851523  16.888635
SI1 0    NaN                  NaN  46.071068  14.693952           NaN  46.071068  14.693952
SK1 0    NaN                  NaN  48.653333  19.027888           NaN  48.653333  19.027888

[128 rows x 7 columns]
>>> entso.bus2sub
            sub_id  interconnect
bus_id                          
AL1 0        AL1 0           NaN
AT1 0        AT1 0           NaN
AT1 1        AT1 1           NaN
BA1 0        BA1 0           NaN
BE1 0        BE1 0           NaN
...            ...           ...
SE2 2 hydro  SE2 2           NaN
SE2 3 hydro  SE2 3           NaN
SE2 4 hydro  SE2 4           NaN
SI1 0 hydro  SI1 0           NaN
SK1 0 hydro  SK1 0           NaN

[457 rows x 2 columns]
BainanXia commented 1 year ago

My understanding is that the substations bear the same name and coordinates than the load buses and and all buses are connected to a substation. Therefore you get the mapping with:

>>> from powersimdata import Grid
>>> entso = Grid("Europe", source="europe_tub", reduction=128)
Title: PyPSA-Eur: An Open Optimisation Model of the European Transmission System (Dataset)
Publication date: 2022-09-20
Version: v0.6.1
DOI: 10.5281/zenodo.7251657
networks.zip has been downloaded previously
WARNING:pypsa.io:Importing network from PyPSA version v0.20.0 while current version is v0.21.2. Read the release notes at https://pypsa.readthedocs.io/en/latest/release_notes.html to prepare your network for import.
INFO:pypsa.io:Imported network elec_s_128_ec.nc has buses, carriers, generators, lines, links, loads, storage_units, stores
>>> entso.sub
        name  interconnect_sub_id        lat        lon  interconnect    pypsa_y    pypsa_x
sub_id                                                                                     
AL1 0    NaN                  NaN  41.587568  19.897914           NaN  41.587568  19.897914
AT1 0    NaN                  NaN  47.354096  11.237535           NaN  47.354096  11.237535
AT1 1    NaN                  NaN  47.671625  15.083487           NaN  47.671625  15.083487
BA1 0    NaN                  NaN  43.811367  18.122704           NaN  43.811367  18.122704
BE1 0    NaN                  NaN  51.005779   4.451911           NaN  51.005779   4.451911
...      ...                  ...        ...        ...           ...        ...        ...
SE2 2    NaN                  NaN  63.114629  16.174075           NaN  63.114629  16.174075
SE2 3    NaN                  NaN  67.551535  18.179626           NaN  67.551535  18.179626
SE2 4    NaN                  NaN  59.851523  16.888635           NaN  59.851523  16.888635
SI1 0    NaN                  NaN  46.071068  14.693952           NaN  46.071068  14.693952
SK1 0    NaN                  NaN  48.653333  19.027888           NaN  48.653333  19.027888

[128 rows x 7 columns]
>>> entso.bus2sub
            sub_id  interconnect
bus_id                          
AL1 0        AL1 0           NaN
AT1 0        AT1 0           NaN
AT1 1        AT1 1           NaN
BA1 0        BA1 0           NaN
BE1 0        BE1 0           NaN
...            ...           ...
SE2 2 hydro  SE2 2           NaN
SE2 3 hydro  SE2 3           NaN
SE2 4 hydro  SE2 4           NaN
SI1 0 hydro  SI1 0           NaN
SK1 0 hydro  SK1 0           NaN

[457 rows x 2 columns]

Exactly.

rouille commented 1 year ago

My understanding is that the substations bear the same name and coordinates than the load buses and and all buses are connected to a substation. Therefore you get the mapping with:

>>> from powersimdata import Grid
>>> entso = Grid("Europe", source="europe_tub", reduction=128)
Title: PyPSA-Eur: An Open Optimisation Model of the European Transmission System (Dataset)
Publication date: 2022-09-20
Version: v0.6.1
DOI: 10.5281/zenodo.7251657
networks.zip has been downloaded previously
WARNING:pypsa.io:Importing network from PyPSA version v0.20.0 while current version is v0.21.2. Read the release notes at https://pypsa.readthedocs.io/en/latest/release_notes.html to prepare your network for import.
INFO:pypsa.io:Imported network elec_s_128_ec.nc has buses, carriers, generators, lines, links, loads, storage_units, stores
>>> entso.sub
        name  interconnect_sub_id        lat        lon  interconnect    pypsa_y    pypsa_x
sub_id                                                                                     
AL1 0    NaN                  NaN  41.587568  19.897914           NaN  41.587568  19.897914
AT1 0    NaN                  NaN  47.354096  11.237535           NaN  47.354096  11.237535
AT1 1    NaN                  NaN  47.671625  15.083487           NaN  47.671625  15.083487
BA1 0    NaN                  NaN  43.811367  18.122704           NaN  43.811367  18.122704
BE1 0    NaN                  NaN  51.005779   4.451911           NaN  51.005779   4.451911
...      ...                  ...        ...        ...           ...        ...        ...
SE2 2    NaN                  NaN  63.114629  16.174075           NaN  63.114629  16.174075
SE2 3    NaN                  NaN  67.551535  18.179626           NaN  67.551535  18.179626
SE2 4    NaN                  NaN  59.851523  16.888635           NaN  59.851523  16.888635
SI1 0    NaN                  NaN  46.071068  14.693952           NaN  46.071068  14.693952
SK1 0    NaN                  NaN  48.653333  19.027888           NaN  48.653333  19.027888

[128 rows x 7 columns]
>>> entso.bus2sub
            sub_id  interconnect
bus_id                          
AL1 0        AL1 0           NaN
AT1 0        AT1 0           NaN
AT1 1        AT1 1           NaN
BA1 0        BA1 0           NaN
BE1 0        BE1 0           NaN
...            ...           ...
SE2 2 hydro  SE2 2           NaN
SE2 3 hydro  SE2 3           NaN
SE2 4 hydro  SE2 4           NaN
SI1 0 hydro  SI1 0           NaN
SK1 0 hydro  SK1 0           NaN

[457 rows x 2 columns]

Exactly.

All the buses belong to one and only one load zone then:

>>> len(set(entso.bus2sub.sub_id))
128

so we should be fine.

BainanXia commented 1 year ago

@rouille So we already have zone_id for all buses in the bus table based on the mappings between load buses and substations?

rouille commented 1 year ago

@rouille So we already have zone_id for all buses in the bus table based on the mappings between load buses and substations?

Yes, you can do that:

>>> entso.bus
             type  Pd  Qd  Gs  Bs  zone_id  ...  pypsa_type  pypsa_unit  pypsa_v_mag_pu_set  pypsa_v_mag_pu_min  pypsa_v_mag_pu_max  pypsa_sub_network
bus_id                                      ...                                                                                                       
AL1 0           3 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 1           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BA1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BE1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
...           ...  ..  ..  ..  ..      ...  ...         ...         ...                 ...                 ...                 ...                ...
SE2 2 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 3 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 4 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SI1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SK1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN

[457 rows x 32 columns]
>>> entso.model_immutables.zones["loadzone2country"][entso.bus2sub.loc["SK1 0 hydro", "sub_id"]]
'Slovakia'
>>> entso.model_immutables.zones["loadzone2interconnect"][entso.bus2sub.loc["BE1 0", "sub_id"]]
'ContinentalEurope'
BainanXia commented 1 year ago

@rouille So we already have zone_id for all buses in the bus table based on the mappings between load buses and substations?

Yes, you can do that:

>>> entso.bus
             type  Pd  Qd  Gs  Bs  zone_id  ...  pypsa_type  pypsa_unit  pypsa_v_mag_pu_set  pypsa_v_mag_pu_min  pypsa_v_mag_pu_max  pypsa_sub_network
bus_id                                      ...                                                                                                       
AL1 0           3 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 1           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BA1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BE1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
...           ...  ..  ..  ..  ..      ...  ...         ...         ...                 ...                 ...                 ...                ...
SE2 2 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 3 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 4 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SI1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SK1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN

[457 rows x 32 columns]
>>> entso.model_immutables.zones["loadzone2country"][entso.bus2sub.loc["SK1 0 hydro", "sub_id"]]
'Slovakia'
>>> entso.model_immutables.zones["loadzone2interconnect"][entso.bus2sub.loc["BE1 0", "sub_id"]]
'ContinentalEurope'

Yes, I understand we have the mappings in model_immutables set properly via sub_id. I'm referring to the zone_id column of the bus table being NaN. The zone_id column of other tables, such as plant relies on the one in the bus table, see here.

rouille commented 1 year ago

@rouille So we already have zone_id for all buses in the bus table based on the mappings between load buses and substations?

Yes, you can do that:

>>> entso.bus
             type  Pd  Qd  Gs  Bs  zone_id  ...  pypsa_type  pypsa_unit  pypsa_v_mag_pu_set  pypsa_v_mag_pu_min  pypsa_v_mag_pu_max  pypsa_sub_network
bus_id                                      ...                                                                                                       
AL1 0           3 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 1           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BA1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BE1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
...           ...  ..  ..  ..  ..      ...  ...         ...         ...                 ...                 ...                 ...                ...
SE2 2 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 3 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 4 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SI1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SK1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN

[457 rows x 32 columns]
>>> entso.model_immutables.zones["loadzone2country"][entso.bus2sub.loc["SK1 0 hydro", "sub_id"]]
'Slovakia'
>>> entso.model_immutables.zones["loadzone2interconnect"][entso.bus2sub.loc["BE1 0", "sub_id"]]
'ContinentalEurope'

Yes, I understand we have the mappings in model_immutables set properly via sub_id. I'm referring to the zone_id column of the bus table being NaN. The zone_id column of other tables, such as plant relies on the one in the bus table, see here.

Sorry I misunderstood. You are right, we need to add zone_id to bus, zone_id/zone_name to plant, from_zone_id/to_zone_id/from_zone_name/to_zone_name to branch and from_interconnect/to_interconnect to dcline. It can be done in this PR or in a following one. What do you prefer?

BainanXia commented 1 year ago

@rouille So we already have zone_id for all buses in the bus table based on the mappings between load buses and substations?

Yes, you can do that:

>>> entso.bus
             type  Pd  Qd  Gs  Bs  zone_id  ...  pypsa_type  pypsa_unit  pypsa_v_mag_pu_set  pypsa_v_mag_pu_min  pypsa_v_mag_pu_max  pypsa_sub_network
bus_id                                      ...                                                                                                       
AL1 0           3 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
AT1 1           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BA1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
BE1 0           1 NaN NaN NaN NaN      NaN  ...                    None                 1.0                 0.0                 inf                   
...           ...  ..  ..  ..  ..      ...  ...         ...         ...                 ...                 ...                 ...                ...
SE2 2 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 3 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SE2 4 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SI1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN
SK1 0 hydro     1 NaN NaN NaN NaN      NaN  ...         NaN         NaN                 NaN                 NaN                 NaN                NaN

[457 rows x 32 columns]
>>> entso.model_immutables.zones["loadzone2country"][entso.bus2sub.loc["SK1 0 hydro", "sub_id"]]
'Slovakia'
>>> entso.model_immutables.zones["loadzone2interconnect"][entso.bus2sub.loc["BE1 0", "sub_id"]]
'ContinentalEurope'

Yes, I understand we have the mappings in model_immutables set properly via sub_id. I'm referring to the zone_id column of the bus table being NaN. The zone_id column of other tables, such as plant relies on the one in the bus table, see here.

Sorry I misunderstood. You are right, we need to add zone_id to bus, zone_id/zone_name to plant, from_zone_id/to_zone_id/from_zone_name/to_zone_name to branch and from_interconnect/to_interconnect to dcline. It can be done in this PR or in a following one. What do you prefer?

🙏 Either way. We can merge this first then.