vinault commented 1 year ago

Describe the current behavior

We are looking for a dataset to do some experimentations with the new Powsybl backend. Pandapower has one here : https://pandapower.readthedocs.io/en/v2.3.0/networks/power_system_test_cases.html#case-2848rte.

@BDonnot : Did you already try out some things with grid2op on a RTE case ? Do you have some advice for us (which case, what/which chronics) ? Thank you very much !

Describe the expected behavior

We want to showcase the whole pipeline with the Powsybl backend on the French grid.

Describe the motivation

No response

Extra Information

No response

vinault commented 1 year ago

@marota

BDonnot commented 1 year ago

Hello,

There are a few snapshots of the French grid open sourced in matpower a few years ago, for example available in pandapower at https://pandapower.readthedocs.io/en/v2.13.1/networks.html#networks:

I used some of these to make benchmark for lightsim2grid by "inventing" some time series from the snapshot. Unfortunately we cannot release real time series for the French grid yet. The legal process would probably last too long.

You can have a look at this script that embed the "time series generator": https://github.com/BDonnot/lightsim2grid/blob/master/benchmarks/benchmark_grid_size.py

marota commented 1 year ago

We start with the 1888-rte generating time-series with the script from Benjamin (@vinault give it a try and let us know).

For IIDM file format, the Pegase network has been extensively used during ITESLA project and can be reuse for test. We will provide a conversion of matpower french network into xiidm. We will see if we can openly share a native xiidm french network file. Maybe look for the situations that have been openly released 10 years ago in matpower: do we have a native xiidm file for that ?

If we want to reduce the size of the grid for experiments and development, this will be something to define

tschuppr commented 1 year ago

Do you know if it is possible to create a prods_charac.csv and grid_layout.json on the go starting from a .json case file, or if some of those files exist in a repo somwhere for the 1888-rte case? @BDonnot @marota

BDonnot commented 1 year ago

Hello,

A grid file is a static description of what the state of the grid is currently, power absorbed, consumed, the flows etc. etc. It has no information in general about how the energy is produced or at which cost etc. Only the amount and location matters (basically).

The "prods_charac.csv" describes, for each generator what are its minimum, maximum values, what are the cost of the fuel, how the production can vary between a certain amount of time etc.

So no it's not possible to create "a" prod_charac.csv file from a grid file in general. Or rather it is possible to create an infinity of them that would be consistent with the grid at hand.

There is no official prods_charac.csv file for any of the grid in matpower. If you want one you have to invent it (ie chose one from the infinity possible). You can for example say that all generators are "thermal", assign max_ramp_up = max_ramp_down , chose some max_p for all generators etc.

I am not aware of anyone having done this work at RTE at the moment.

marota commented 1 year ago

As Benjamin mentionned, some info are missing in the grid.json file as we are running scenarios over time. In particular the costs and the ramps of the generators.

Here are some typical numbers we used to far for the IEEE grids

marginal_price = { 'wind': 0, 'solar': 0, 'nuclear': 35, 'hydro': 36,#20 hydro price are usually low but here we make it slightly higher than nuclear to generate good nuclear and hydro profiles with our dispatch that does not take really into account hydro as a reservoir 'biomass': 45, 'naturalgas': 50, 'biogas': 45, 'geothermal': 40, 'coal': 50, 'oil': 100, }

Ramps here are normalized by pu and every 5 min

nuclear_ramps = {'nuclear01': 100 / (1200 * 12)}

coal_steam_turbine_ramps = {'coal01': 103 / (574 12), 'coal02': 151 / (845 12), }

Slow ramps

gas_steam_turbine_ramps = {'gast01': 55 / (237 12), 'gast02': 123 / (521 12), }

Medium ramps

gas_combined_cycle_ramps = {'gascc01': 84 / (264 12), 'gascc02': 161 / (506 12), 'gascc03': 265 / (835 * 12), }

Fast ramps

gas_turbine_ramps = {'gasgt01': 16 / (47 12), 'gasgt02': 16 / (48 12), 'gasgt03': 23 / (66 12), 'gasgt04': 29 / (86 12), }

ramp_hydro = 0.5 * p_max_hydro / 12

marota commented 1 year ago

Those numbers were taken from reference from there (given by Carlo Brancucci (when he was at NREL) ExamplesGeneratorsProperties.pdf

marota commented 1 year ago

Regarding the grid_layout.json, there are no coordinates available from the open rte cases (as there was some anonymization in the process). So the best that can probably be done is to plot a graph with some graph layout algorithm and extract the coordinates of the nodes to create this grid_layout.json,

What you get from pandapower graph layout coordinates

from plt.simple_plot(net,bus_size=.1,trafo_size=.1)

marota commented 1 year ago

For loadflow issues on the 1888 rte case, this could be due to some issue in per uniting in the pandapower cases. Normally if you load a grid in pypowsybl and run a load flow with it, it converges properly. Should try this to start with @yojvr

When you load the 1888 rte case, are you currently loading the grid.mat or grid.json ?

marota commented 1 year ago

When loading the grid.mat with pypowsybl, convergence is fine (and cumulated load is 59GW):

import pypowsybl as pp
n =  pp.network.load("YourPath/src/data_test/Test_1888rte/grid.mat")
results = pp.loadflow.run_ac(n)
print(results)#check convergence
print(n.get_loads()["p"].sum())#check load level

When loading this .mat with grid2op and for a load level of 56GW in the first step of the chronics, there is indeed a divergence:

env = grid2op.make("src/data_test/Test_1888rte",
                   grid_path="YourPath/src/data_test/Test_1888rte/grid.mat",
                   backend=PowsyblBackend(detailed_infos_for_cascading_failures=False),
                   _add_to_name="stradded"
                   )

However when doing it on the best case without chronics, it converges again:

from grid2op.Chronics import ChangeNothing
env = grid2op.make("src/data_test/Test_1888rte",
                   grid_path="YourPath/src/data_test/Test_1888rte/grid.mat",
                   # "src\data_test\Test_1888rte",
                   backend=PowsyblBackend(detailed_infos_for_cascading_failures=False),
                   _add_to_name="stradded",
                   chronics_class=ChangeNothing
                   # backend=PandaPowerBackend(),
                   #param=p
                   )

So this probably a problem of convergence for the value of the Chronic in the first step

WARNING: comparing the flows computed from the grid.mat with pypowsybl and from the grid.json using lightsim2grid, there are some notable differences in power flows (here examples of lines with more than 100% difference, so this is probably not just a matter of different conventions between the two lines extremities for instance):

obs_simu,*_=obs.simulate(env.action_space({}),time_step=0)
df=pd.DataFrame({"lightSimFlow":np.abs(obs_simu.p_or),"powsyblFlow":n.get_lines()["p1"]})
df["diff"]=df["lightSimFlow"]-df["powsyblFlow"]
df["relative_diff"]=df["diff"].abs()/df["lightSimFlow"]
df.describe()

Side note negative loads in .mat (as exposed by pypowsybl) seem to correspond to s_gen in grid.json (as exposed by pandapower)

marota commented 1 year ago

The test chronic has the following load level over a day

marota commented 1 year ago

To get a scenario that runs, I suggest we first run things with DC powerflows on the generated chronics @yojvr So add this option in the backend, which should use env parameter "p.ENV_DC=True" to get run

marota commented 1 year ago

Here are chronics that partially converge in AC power flow over the scenario. Convergence periods are so far:

timesteps 0 to 5
timesteps 83 until 111
timesteps 167 to 220
timestep 237 à 287

So at least you can do grid2op.make() without non converging load flow error initially.

When generating the chronics, I used the .mat file, and made sure not too touch negative loads: their value is constant and taken from the original file. These negative loads in this format corresponds to the sgen in pandapower format. This is the code addition in get_loads_gens:

    #deal with negative loads and don't change their values as it might be a bit tricky
    if neg_load_no_change :
        idx_negative_load=np.where(load_p_init<0)[0]
        load_p[:, idx_negative_load] = load_p_init[idx_negative_load]
        load_q[:, idx_negative_load] = load_q_init[idx_negative_load]

It seems not to be converging in particular when the load is the lowest from midnight to 6-7am.

Making the timesteps 221 to 236 converge would already give an interesting continuous period on this grid.

load_p_q_prod_p.zip

marota commented 1 year ago

Script pour tester les convergences sur la chronique rtecase_loading.zip

marota commented 1 year ago

Converging when total load remains within 94%-106% of initial grid loading. Converging after 6-7am all day with that. Here is the resulting load curve without changing morning night loading (which are still not converging)

marota commented 1 year ago

When changing the load factor directly with pypowsybl, I get convergence from 50% to 150% from the initial loading. So something is probably not set properly in the pypowsybl-grid2op backend as I would expect the same range of convergence.

See this script for this experiment


import pypowsybl as pp
import pandas as pd
import numpy as np

#script to test that varying the loading on the grid between 50% and 150% always make a converging power flow case

def change_loading_pp_network(net_path, loading_factor=1.0):
    # init network
    n = pp.network.load(net_path)
    results = pp.loadflow.run_ac(n)
    flow_init = n.get_lines()["p1"]

    # update loads
    df_loads_init = n.get_loads()
    load_col_names = df_loads_init.columns
    df_loads_init[load_col_names[2:6]] = loading_factor * df_loads_init[load_col_names[2:6]]
    n.update_loads(df_loads_init[load_col_names[2:6]])

    print("total load is: " + str(df_loads_init.p0.sum()))

    # update generators
    df_gens_init = n.get_generators()
    init_target_v = df_gens_init.target_v.copy(deep=True)
    init_target_q = df_gens_init.target_q.copy(deep=True)
    df_gens_init.target_p = loading_factor * df_gens_init.target_p
    n.update_generators(df_gens_init[["target_p"]])

    print("total prod target is: " + str(df_gens_init.target_p.sum()))
    # print("total prod is: " + str(df_gens_init.p.sum()))

    has_converged = False
    cum_voltage_factor = 1.
    while not has_converged:
        results = pp.loadflow.run_ac(n)
        print(results[0].status)

        if (results[0].status.value == 0):#in case it has converged without changing voltages
            has_converged = True
            print("it has converged")
            print("cumulated voltage factor update is: " + str(cum_voltage_factor))
            flow_end = n.get_lines()["p1"]
            print(np.mean(np.abs((flow_init - flow_end) / (flow_init + 1e-5))))
        else:
            #print(results[0].status)
            #print(results[0])
            print("divergence, changig target voltage")
            if loading_factor < 1.0:
                cum_voltage_factor -= 0.001
                print("cumulated voltage factor is: " + str(cum_voltage_factor))
                df_gens_init.target_v = cum_voltage_factor * df_gens_init.target_v
                df_gens_init.target_q = cum_voltage_factor * init_target_q
            else:
                cum_voltage_factor += 0.001
                print("cumulated voltage factor is: " + str(cum_voltage_factor))
                df_gens_init.target_v = cum_voltage_factor * init_target_v
                df_gens_init.target_q = cum_voltage_factor * init_target_q
                n.update_generators(df_gens_init[["target_v"]])

    return n

# check convergence on original case with pypowsybl first
net_file_path = "YourPath/pypowsybl-grid2opbackend/src/data_test/Test_1888rte/grid.mat"
n = pp.network.load(net_file_path)  # pp.network.create_ieee1888()#create_ieee14()
# results = pp.loadflow.run_ac(n)
results = pp.loadflow.run_ac(n)

for loading_factor in range(50,150,5):
    print("test convergence with loading factor of: "+str(loading_factor/100)+" %")
    change_loading_pp_network(net_file_path,loading_factor/100)

marota commented 1 year ago

@tschuppr explain here how you obtained chronics convergence to close this issue

tschuppr commented 1 year ago

We obtained convergence by modifying the values of active power generation from negative to positive to comply with pypowsybl vision.

powsybl / pypowsybl-grid2opbackend

MATPOWER French Network #2

Describe the current behavior

Describe the expected behavior

Describe the motivation

Extra Information

Ramps here are normalized by pu and every 5 min

Slow ramps

Medium ramps

Fast ramps