Implement reactive power provision as a decision variable

[floracharbo]

• implement the reactive power decision variable and constraints in the optimisation
• add this as an action variable in the RL environment LocalElecEnv. Convert the action variable into environment update in the Action_translator.
• a first step might be to implement this with aggregate_actions = False

[floracharbo]

In action_translator the two main methods are:

• actions_to_env_vars
• optimisation_to_rl_env_action

see what happens for flexible_store_action and try and do something similar.

initial_processing is where you establish min and max values at each time step. But here we may not be able to define min and max Q a priori without having knowledge of actual P value.

[floracharbo]

for actions_to_env_vars, I suggest we explore doing:

1. translate flexible_store_action into charge/discharge (=P)
2. define the min and max bounds for Q based on P^2 + Q^2 <= S^2
3. translate reactive_power_action into Q.

[floracharbo]

for optimisation_to_rl_env_action, I suggest we explore doing:

1. translate the charge/discharge (=P) into flexible_store_action
2. define the min and max bounds for Q based on P^2 + Q^2 <= S^2
3. translate Q into reactive_power_action.

[floracharbo]

The decision variable could be

• reactive_power_action : no import or export of reactive power
• reactive_power_action -1 max export Q<0
• reactive_power_action 1 max import Q>0

Or you may thing of other definitions: e.g. reactive_power_action = Q/P ? Q/S? angle? cos(angle)? etc etc

[julie-vienne]

for optimisation_to_rl_env_action, I suggest we explore doing:

1. translate the charge/discharge (=P) into flexible_store_action
2. define the min and max bounds for Q based on P^2 + Q^2 <= S^2
3. translate Q into reactive_power_action.

Have I understood correctly that we could be following these steps to consider when the battery would be available to provide reactive power? I.e. if we directly translate res['q_car_flex'] from the optimization, we might be using flexibility when the car is not available?

[floracharbo]

for optimisation_to_rl_env_action, I suggest we explore doing:

1. translate the charge/discharge (=P) into flexible_store_action
2. define the min and max bounds for Q based on P^2 + Q^2 <= S^2
3. translate Q into reactive_power_action.

Have I understood correctly that we could be following these steps to consider when the battery would be available to provide reactive power? I.e. if we directly translate res['q_car_flex'] from the optimization, we might be using flexibility when the car is not available?

Yes for 1-2-3;

For res['q_car_flex'], the optimisation should set this to 0 when the car is not available. charge and discharge_other have to be zero when the car available, so also p_car_flex, so also q_car_flex when pf >=0. We should add a constraint to encore q_car_flex = 0 when the car is not available now that pf will be freely varying and may take a value of 0.

[floracharbo]

e.g. the constraint for charge was

  p.add_constraint(
      charge <= car['batch_avail_car'][:, 0: self.N] * self.syst['M']
  )

here because q_car_flex can be positive or negative, maybe we can break it down into its positive and its negative component, and then enforce this constraint on both?

[julie-vienne]

e.g. the constraint for charge was

  p.add_constraint(
      charge <= car['batch_avail_car'][:, 0: self.N] * self.syst['M']
  )

here because q_car_flex can be positive or negative, maybe we can break it down into its positive and its negative component, and then enforce this constraint on both?

That's a very good point, yes. Since we anyway need q_car_flex2 >= q_car_flex**2, we could for example enforce that

p.add_constraint(q_car_flex2 <= car['batch_avail_car'][:, 0: self.N] * self.syst['M'])

If car['batch_avail_car'][:, t] * self.syst['M'] = 0, this constraint makes sure that q_car_flex = 0 and if it's 1000000.0, then q_car_flex can be either a large positive or a large negative, constrained again by the charge and discharge power.

[julie-vienne]

In _flex_store_actions of the action translators, I'm not sure why the flexible_store_actions in the case of a discharge is defined using res[charge] (line 6-9 below).

        for home in range(self.n_homes):
            if store_bool_flex[home]:
                if abs(res['discharge_other'][home, time_step]) < 1e-3 \
                        and abs(res['charge'][home, time_step]) < 1e-3:
                    flexible_store_actions[home] = 0
                elif res['discharge_other'][home, time_step] > 1e-3:
                    flexible_store_actions[home] = \
                        (self.min_discharge[home] - res['charge'][home, time_step]) \
                        / (self.min_discharge[home] - self.max_discharge[home])
                else:
                    if abs(res['charge'][home, time_step] - self.max_charge[home]) < 1e-3:
                        flexible_store_actions[home] = 1
                    else:
                        flexible_store_actions[home] = (
                            res['charge'][home, time_step] - self.min_charge[home]
                        ) / (self.max_charge[home] - self.min_charge[home])

In comparison, our reactive power export should rather be based on the discharge rather than the charge, right?

[floracharbo]

In _flex_store_actions of the action translators, I'm not sure why the flexible_store_actions in the case of a discharge is defined using res[charge] (line 6-9 below).

        for home in range(self.n_homes):
            if store_bool_flex[home]:
                if abs(res['discharge_other'][home, time_step]) < 1e-3 \
                        and abs(res['charge'][home, time_step]) < 1e-3:
                    flexible_store_actions[home] = 0
                elif res['discharge_other'][home, time_step] > 1e-3:
                    flexible_store_actions[home] = \
                        (self.min_discharge[home] - res['charge'][home, time_step]) \
                        / (self.min_discharge[home] - self.max_discharge[home])
                else:
                    if abs(res['charge'][home, time_step] - self.max_charge[home]) < 1e-3:
                        flexible_store_actions[home] = 1
                    else:
                        flexible_store_actions[home] = (
                            res['charge'][home, time_step] - self.min_charge[home]
                        ) / (self.max_charge[home] - self.min_charge[home])

In comparison, our reactive power export should rather be based on the discharge rather than the charge, right?

That looks like a mistake!! Thanks for pointing that out. I will fix this. Just doing a run now to check things.

[julie-vienne]

@floracharbo so far, if no charge or discharge is being used, there is also no reactive power available

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        # if no charge or discharge, no reactive power either
        if indiv_flexible_store_action == 0:
            indiv_flexible_q_car = 0

However, it might be the case that we don't want to charge or discharge the battery but still provide reactive power for voltage control. Do you think we should allow our agents to do this, i.e. if indiv_flexible_store_action == 0, then indiv_flexible_q_car has a full range of flexibility (= -1 can provide full export).

[floracharbo]

@floracharbo so far, if no charge or discharge is being used, there is also no reactive power available

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        # if no charge or discharge, no reactive power either
        if indiv_flexible_store_action == 0:
            indiv_flexible_q_car = 0

However, it might be the case that we don't want to charge or discharge the battery but still provide reactive power for voltage control. Do you think we should allow our agents to do this, i.e. if indiv_flexible_store_action == 0, then indiv_flexible_q_car has a full range of flexibility (= -1 can provide full export).

Yes I would say so! They can still export or import reactive power within the apparent power constraint in the same way.

[julie-vienne]

@floracharbo so far, if no charge or discharge is being used, there is also no reactive power available

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        # if no charge or discharge, no reactive power either
        if indiv_flexible_store_action == 0:
            indiv_flexible_q_car = 0

However, it might be the case that we don't want to charge or discharge the battery but still provide reactive power for voltage control. Do you think we should allow our agents to do this, i.e. if indiv_flexible_store_action == 0, then indiv_flexible_q_car has a full range of flexibility (= -1 can provide full export).

Yes I would say so! They can still export or import reactive power within the apparent power constraint in the same way.

In that case, do we give them full positive or negative flexibility? Do you see a way to allow them to have flexibility in both directions or do we have to assign only one value to indiv_flexible_q_car?

[floracharbo]

I don't understand your question. Their q_min will be -S, their q_max = S, then if action <= 0 if will scale between 0 and q_min, if action >=0 it scales between 0 and q_max, just as usual?

[julie-vienne]

_calculate_flexible_q_car

Sorry, I think I got confused because of the naming of my variables. I think this should do the trick. If they decide to import, we calculate what they can use a flexibility using charge. If charge is zero, they have the full positive flexibility available.

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        if indiv_flexible_q_car_action > 0:
            charge = indiv_flexible_store_action
            max_q_car_import = np.sqrt(self.max_apparent_power_car**2 - charge**2)
            indiv_flexible_q_car = (
                indiv_flexible_q_car_action - self.min_q_car_import
                    ) / (max_q_car_import - self.min_q_car_import)
        elif indiv_flexible_q_car_action < 0:
            discharge = indiv_flexible_store_action
            max_q_car_export = - np.sqrt(self.max_apparent_power_car**2 - discharge**2)
            indiv_flexible_q_car = (self.min_q_car_export - indiv_flexible_q_car_action) \
                / (self.min_q_car_export - max_q_car_export)
        else:
            indiv_flexible_q_car = 0

        return indiv_flexible_q_car

[floracharbo]

_calculate_flexible_q_car

Sorry, I think I got confused because of the naming of my variables. I think this should do the trick. If they decide to import, we calculate what they can use a flexibility using charge. If charge is zero, they have the full positive flexibility available.

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        if indiv_flexible_q_car_action > 0:
            charge = indiv_flexible_store_action
            max_q_car_import = np.sqrt(self.max_apparent_power_car**2 - charge**2)
            indiv_flexible_q_car = (
                indiv_flexible_q_car_action - self.min_q_car_import
                    ) / (max_q_car_import - self.min_q_car_import)
        elif indiv_flexible_q_car_action < 0:
            discharge = indiv_flexible_store_action
            max_q_car_export = - np.sqrt(self.max_apparent_power_car**2 - discharge**2)
            indiv_flexible_q_car = (self.min_q_car_export - indiv_flexible_q_car_action) \
                / (self.min_q_car_export - max_q_car_export)
        else:
            indiv_flexible_q_car = 0

        return indiv_flexible_q_car

if charge = discharge = res['ds'] = 0, the reactive power has full flexibility to be both positive or negative, right? not just positive?

[julie-vienne]

_calculate_flexible_q_car

Sorry, I think I got confused because of the naming of my variables. I think this should do the trick. If they decide to import, we calculate what they can use a flexibility using charge. If charge is zero, they have the full positive flexibility available.

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        if indiv_flexible_q_car_action > 0:
            charge = indiv_flexible_store_action
            max_q_car_import = np.sqrt(self.max_apparent_power_car**2 - charge**2)
            indiv_flexible_q_car = (
                indiv_flexible_q_car_action - self.min_q_car_import
                    ) / (max_q_car_import - self.min_q_car_import)
        elif indiv_flexible_q_car_action < 0:
            discharge = indiv_flexible_store_action
            max_q_car_export = - np.sqrt(self.max_apparent_power_car**2 - discharge**2)
            indiv_flexible_q_car = (self.min_q_car_export - indiv_flexible_q_car_action) \
                / (self.min_q_car_export - max_q_car_export)
        else:
            indiv_flexible_q_car = 0

        return indiv_flexible_q_car

if charge = discharge = res['ds'] = 0, the reactive power has full flexibility to be both positive or negative, right? not just positive?

Yes sorry, I was not super clear. I think what these line do is

• if q_car_action > 0 and charge = 0 --> indiv_flexible_q_car = 1
• if q_car_action > 0 and charge > 0 --> indiv_flexible_q_car between 0 and 1, depending on charge
• if q_car_action < 0 and discharge = 0 --> indiv_flexible_q_car = -1
• if q_car_action < 0 and discharge > 0 --> indiv_flexible_q_car between -1 and 0, depending on discharge

[floracharbo]

_calculate_flexible_q_car

Sorry, I think I got confused because of the naming of my variables. I think this should do the trick. If they decide to import, we calculate what they can use a flexibility using charge. If charge is zero, they have the full positive flexibility available.

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        if indiv_flexible_q_car_action > 0:
            charge = indiv_flexible_store_action
            max_q_car_import = np.sqrt(self.max_apparent_power_car**2 - charge**2)
            indiv_flexible_q_car = (
                indiv_flexible_q_car_action - self.min_q_car_import
                    ) / (max_q_car_import - self.min_q_car_import)
        elif indiv_flexible_q_car_action < 0:
            discharge = indiv_flexible_store_action
            max_q_car_export = - np.sqrt(self.max_apparent_power_car**2 - discharge**2)
            indiv_flexible_q_car = (self.min_q_car_export - indiv_flexible_q_car_action) \
                / (self.min_q_car_export - max_q_car_export)
        else:
            indiv_flexible_q_car = 0

        return indiv_flexible_q_car

if charge = discharge = res['ds'] = 0, the reactive power has full flexibility to be both positive or negative, right? not just positive?

why indiv_flexible_q_car= -1or 1? I thought indiv_flexible_q_car was the amount of reactive power exported or imported in kVAR

[julie-vienne]

_calculate_flexible_q_car

Sorry, I think I got confused because of the naming of my variables. I think this should do the trick. If they decide to import, we calculate what they can use a flexibility using charge. If charge is zero, they have the full positive flexibility available.

    def _calculate_flexible_q_car(self, indiv_flexible_store_action, indiv_flexible_q_car_action):
        if indiv_flexible_q_car_action > 0:
            charge = indiv_flexible_store_action
            max_q_car_import = np.sqrt(self.max_apparent_power_car**2 - charge**2)
            indiv_flexible_q_car = (
                indiv_flexible_q_car_action - self.min_q_car_import
                    ) / (max_q_car_import - self.min_q_car_import)
        elif indiv_flexible_q_car_action < 0:
            discharge = indiv_flexible_store_action
            max_q_car_export = - np.sqrt(self.max_apparent_power_car**2 - discharge**2)
            indiv_flexible_q_car = (self.min_q_car_export - indiv_flexible_q_car_action) \
                / (self.min_q_car_export - max_q_car_export)
        else:
            indiv_flexible_q_car = 0

        return indiv_flexible_q_car

if charge = discharge = res['ds'] = 0, the reactive power has full flexibility to be both positive or negative, right? not just positive?

why indiv_flexible_q_car= -1or 1? I thought indiv_flexible_q_car was the amount of reactive power exported or imported in kVAR

yes, you're right. The action between -1 and 1 is actually performed in the function _flex_q_car_actions and indiv_flexibleq_car are indeed the values in kVAR. Sorry for the confusion, I believe it now makes more sense in the code and the variable names are also more coherent.