ConstControl for several load busses

[MagicPatze]

Hello Pandapower-Team,

currently i am trying to implement a time series much like your time_series tutorial. Sadly I am having some issues with setting a Const Controller.

Like in your tutorial, I am creating two DFDatas, only that my input for that are arrays. These arrays have timesteps as rows and all the load indexes from the network (I am using Kerber Networks) as columns.

In your documentation, it says that it is possible to create one Controller for several loads (and gens) aswell. Can you explain to me, how I do that?

When creating I dont know what to type in for the needed inputs. net: is my network, thats clear. element: is that the element of my network? I typed 'sgen' and 'load' like in the tutorial. variable = p_mw, clear. element_index: right now i wrote "net.load.index" for both, as by default Kerber networks only have loads and i simulate a load and a gen at those busses, that contain a load. Or do I have to negate my gen profiles from e.g. 10 to -10? data_source: clear as well profile_name: That is my biggest question. What do I have to type in here? When i insert a list of node integers, it creates an error later on, cause it does not accept a list. Note that my DFData does not have columns with names, like in the tutorial, but only integers from 1 to e.g. 14.

Or would it be better to create multiple controllers with a loop over all my nodes?

Thank you so much for help in advance!

[FlorianShepherd]

Hello MagicPatze,

atm we're refactoring a lot in the timeseries module + control. Have you pulled the current develop branch and looked at the current documentation?

See: https://pandapower.readthedocs.io/en/latest/timeseries/timeseries_example.html for example.

The data_source is a DataFrame with index = timesteps and columns = index of the elements (for example). It must have the same length of columns as len(element_index)

[rbolgaryn]

Hi MagicPatze,

the profile_name refers to the column names in the DataFrame of the DataSource, and it should be able to accept a list.

There should be 1 DataSource for 1 ConstControl, the DataSource should have all the profiles for all the loads you want to control as columns.

You can read more about the signs of the p_mw values in the documentation about the update from pandapower 1 to pandapower 2:

https://pandapower.readthedocs.io/en/latest/about/update.html

Creating controllers in a loop separately could lead to a slower time series calculation, but it depends on your use-case whether or not it matters for you. The results would be the same.

[MagicPatze]

Thank you both for your answers!

I temporarilly solved the issue by creating several controllers; one for ever single bus. Since this evokes some critical runtime issues though, for networks containing more than 100 busses, I want to insert a list into "profile_name". My list, named "nodes" contains one number for each bus, being: nodes = [0, 1, 2, 3, 4, 5, ...] When typing ConstControl(net, ..., profile_name= nodes), i get the following error later on, when the output writer appears in the code: KeyError: "None of [Int64Index([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], dtype='int64')] are in the [index]"

Then i thought the problem might be, that DFData's columns are strings in place of ints, so I converted my list of ints into a list of strings, but I get a similar error.

Is it possible for you, to diagnose a mistake there? I know for sure, that profile_name is the source of my error, because creating several controllers worked for me, it is just really slow.

[rbolgaryn]

Hi MagicPatze,

can you write a minimal example for the issue?

[rbolgaryn]

Also, the data for the profiles is accessed in the code with:

data_source.df.loc[time_step, profile_name]

You can try accessing the data directly from the DataFrame that you used to create the data_source, using profile_name, to see if it works or what kind of errors you are getting.

[MagicPatze]

Alright, so what I did is I retrieved loads and gens from another python script, wrote them into an array. Then I subtracted one from the other, so what I have is a total load/gen: dsTotal = DFData(profilesTotal) When accessing the df of my dsTotal I see the proper values for my gens/loads, columns being the number of nodes and rows being my timesteps

Then i define my create_controllers function: def create_controllers(net, dsInj, dsTotal,nodes): ConstControl(net,element='load',variable='p_mw',element_index=net.load.index,data_source=dsTotal, profile_name=nodes)

I dont know what else could be helfull for you as an example.

When I print nodes, I get the following: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] Of course, the size of nodes depends on the net I used. For this example I used a small Kerber network.

Edit: When accessing and/or printing the dataframe I used dsTotal.df.loc[timesteps,nodes] which did not evoke an error.

[MagicPatze]

Also, when typing: def create_controllers(net, dsTotal): ConstControl(net, element='load', variable='p_mw', element_index=net.load.index,data_source=dsTotal, profile_name=dsTotal.df.columns) I get the following error: ValueError: Must have equal len keys and value when setting with an iterable

I dont know why this is occurs, though.

[MagicPatze]

So I am running into another problem as well: When I create dsTotal, which adds load and gen data (load being positive and gen negative), the voltage magnitude gets reduced, even when the gen is bigger than the load. Is that even possible? Or can it be that the timeseries can not handle negative loads?

[rbolgaryn]

Hi MagicPatze,

can you try running the following code and see if it works?

`

import pandapower.networks

import pandapower.control

net = pp.networks.create_kerber_landnetz_freileitung_1()

profiles = pd.DataFrame(columns=net.load.index, data=0.01*np.random.rand(10, len(net.load)))

ds = pp.timeseries.DFData(df=profiles)

pp.control.ConstControl(net, element='load', variable='p_mw', element_index=net.load.index,
                        profile_name=profiles.columns, data_source=ds)

pp.timeseries.run_timeseries(net, time_steps=range(10))

net.output_writer.at[0, 'object'].output['res_line.loading_percent'].max(axis=1).plot()

`

Can you reproduce the issue in a similar way?

[MagicPatze]

If i only run your code on its own, it gives me an attribute error: AttributeError: 'pandapowerNet' instance has no attribute 'output_writer'

which I understand, because no ow is defined there.

[MagicPatze]

So, I actually found my mistake at inserting a list into the ConstControl: In my list "nodes", the two busses 0 and 1 at the trafo were still included. Now my problem is, that the voltages do not differ in any way... I will show you my code:

`import os import tempfile

import pandapower as pp import pandas as pd import numpy as np

from pandapower.timeseries.data_sources.frame_data import DFData from pandapower.timeseries.output_writer import OutputWriter from pandapower.timeseries.run_time_series import run_timeseries from pandapower.control.controller.const_control import ConstControl

def timeseries_each_day(output_dir, net, timesteps, d, powInjRet, powSubtrRet, gridnodes):

#create new variable, because pandapower only uses time_steps in stead of timesteps
time_steps = timesteps

#retrieve data source
profilesInj, profilesSubtr, dsInj, dsSubtr = retrieve_data_source(timesteps, d, powInjRet, powSubtrRet, gridnodes)

create_controllers(net,dsInj,dsSubtr,gridnodes,profilesInj, profilesSubtr)

#the output writer with the desired results to be stored to files
ow = create_output_writer(net, timesteps, output_dir=output_dir)

#the main time series function
run_timeseries(net, time_steps, output_writer=ow, continue_on_divergence=True)

def retrieve_data_source(timesteps, d, powInjRet, powSubtrRet, gridnodes):

#here I retrieve the arrays of an optimization, which ejects the subtractions and injections of a "house" for every timestep and day
#initialize new arrays the size of wanted array
powInjDay = np.zeros((len(gridnodes),len(timesteps)))
powSubtrDay = np.zeros((len(gridnodes),len(timesteps)))

#new array without axis d (inside for loop for d)
for n in gridnodes:
    for t in timesteps:
        powInjDay[n,t] = powInjRet[d,n,t]
        powSubtrDay[n,t] = powSubtrRet[d,n,t]

profilesPreInj = pd.DataFrame(powInjDay)
profilesPreSubtr = pd.DataFrame(powSubtrDay)

#transpose DataFrame to fit the standard layout of given DataFrame. Afterwards columns are nodes, rows are timesteps
profilesInj = profilesPreInj.transpose()
profilesSubtr = profilesPreSubtr.transpose()

#delete first two columns of dataframe, so dataframe only contains nodes which are connected to load/gen
profilesInj = profilesInj.drop(0,axis=1)
profilesInj= profilesInj.drop(1,axis=1)
profilesSubtr = profilesSubtr.drop(0, axis=1)
profilesSubtr= profilesSubtr.drop(1,axis=1)

#split up profiles in gen(injection) and load(subtraction) profiles, to properly insert them in 2 const_controllers
dsInj = DFData(profilesInj)
dsSubtr = DFData(profilesSubtr)

return profilesInj, profilesSubtr, dsInj, dsSubtr, dsTotal

def create_controllers(net, dsInj, dsSubtr, dsTotal,gridnodes, profilesInj, profilesSubtr):

ConstControl(net, element= 'load', variable= 'p_mw', element_index=net.load.index, data_source=dsSubtr, profile_name=dsSubtr.df.columns)
ConstControl(net, element= 'sgen', variable= 'p_mw', element_index=net.sgen.index, data_source=dsInj, profile_name=dsInj.df.columns)

def create_output_writer(net, timesteps, output_dir): time_steps = timesteps ow = OutputWriter(net, time_steps, output_path=output_dir, output_file_type=".json")

these variables are saved to the harddisk after / during the time series loop

ow.log_variable('res_bus', 'vm_pu')
ow.log_variable('res_line', 'loading_percent')

return ow

#execution follows:

def run_timeloop(net, timesteps, days, powInjRet, powSubtrRet, gridnodes,critical_flag,solution_found):

nodes = {}

nodes["grid"] = net.bus.index.to_numpy()
nodes["trafo"] = net.trafo['lv_bus'].to_numpy()
nodes["load"] = net.load['bus'].to_numpy()
nodes["bat"] = net.load['bus'].to_numpy()

vm_pu_total = {}
for n in gridnodes:
    for d in days:
        for t in timesteps:
            vm_pu_total[n,d,t] = 0

for d in days:
    output_dir = os.path.join(tempfile.gettempdir(), "time_series_example" + str(d))
    print("Results can be found in your local temp folder: {}".format(output_dir))
    if not os.path.exists(output_dir):
        os.mkdir(output_dir)
    timeseries_each_day(output_dir, net, timesteps, d, powInjRet, powSubtrRet, gridnodes)

    #Message to rbolgaryn: I here just read out the voltage magnitude and check if it differs from 400 V for more than 4 percent

    # read out json files for voltages and return time and place of violation
    vm_pu_file = os.path.join(output_dir, "res_bus", "vm_pu.json")
    vm_pu = pd.read_json(vm_pu_file, convert_axes=True)
    # sort dataframe to get timesteps(rows) in the correct order
    vm_pu = vm_pu.sort_index(axis=0)
    # vm_pu was creating keyerror, so changed into array
    vm_pu_final = vm_pu.values
    print("")

    for n in gridnodes:
        for t in timesteps:

            if(vm_pu_final[t,n] < 0.96 or vm_pu_final[t,n] > 1.04):
                if(n in nodes["bat"]):
                    critical_flag[n,d,t] = 1
                    solution_found[d] = False
                    print("voltage violation found for node "+str(n)+" and timestep "+str(t))
            else:
                pass

    if(all((vm_pu_final[t,n] >= 0.96 and vm_pu_final[t,n] <= 1.04) for n in gridnodes for t in timesteps)) == True:
        solution_found[d] = True
        print("solution was found for day" +str(d))

    for n in gridnodes:
        for t in timesteps:
            vm_pu_total[n,d,t] = vm_pu_final[t,n]

vm_pu_total = np.array([[[vm_pu_total[n,d,t] for t in timesteps] for d in days] for n in gridnodes])

return output_dir,critical_flag,solution_found, vm_pu_total`

[MagicPatze]

If you can not understand the long code, chatting privately could also be possible, could it not?

[MagicPatze]

Is it, by any chance, possible that the problem lies within the output writer? Even though I logged the variables vm_pu and p_mw as well, the value of vm_pu at each node is 99,9973 % .

[MagicPatze]

I would be really greatful for some help, its urgent.

[rbolgaryn]

Hi MagicPatze,

I will not have the time to go through your project code and debug it for you. Instead, I was expecting you to create a very minimalistic example with max. 10-15 lines of code that demonstrates the error. Please provide such a minimalistic example, similarly to the code in my previous comment.

Also, try pulling the latest version from branch develop before doing anything else.

Roman

[rbolgaryn]

Dear @MagicPatze, what is the status of the problem?

[rbolgaryn]

Found the issue, the reason is documented in #609 and fixed with pull request #611. Please pull branch develop.