floracharbo / MARL_local_electricity

Multi-agent reinforcement learning for privacy-preserving, scalable residential energy flexibility coordination
GNU Affero General Public License v3.0
22 stars 5 forks source link

constrain reactive power if management is off #40

Closed floracharbo closed 1 year ago

floracharbo commented 1 year ago

make sure that when the management of the reactive power is off, the power factor is constrained to the optimiser cannot 'cheat'.

julie-vienne commented 1 year ago

@floracharbo when constraining p.add_constraint(qi == 0), meaning that the buses cannot provide reactive power and everything must be imported or exported to the grid, the optimization leads to results but they then lead to assert errors between the total rewards of the optimization and the Rl environment or the following error: assert cons_flex[home] <= np.sum(batch_flex[home][h][1:]) + 1e-3. I don't perfectly understand yet why this creates such a difference to the RL simulations as within pandapower, we also have qi = 0

floracharbo commented 1 year ago
floracharbo commented 1 year ago

I am assuming this error occurs when learning from optimisations right? no in pure exploration-based learning.

What is the gap? Is it lower than the tol_cons_constraints?

floracharbo commented 1 year ago

this issue is looked into in #29

julie-vienne commented 1 year ago
  • so firstly if we don't manage reactive power, shouldn't we constrain it to power factor (e.g. 0.95) * p instead of 0?

I indeed see two options here, when do not use reactive power for voltage control:

floracharbo commented 1 year ago

In the normal operation of the power system, there is a power factor which is not zero so I would use that (option 1)

julie-vienne commented 1 year ago

So you would also start adding the corresponding reactive power to our function pf_simulation in addition to the optimizer?

julie-vienne commented 1 year ago

@floracharbo you can have a look at #29 to see how it's been implemented. I'm testing it right now to see if it leads to any issues.

floracharbo commented 1 year ago

So you would also start adding the corresponding reactive power to our function pf_simulation in addition to the optimizer?

yes. Also this means that all along we were not comparing the same thing when comparing the optimisation results and the pp simulations, as reactive power was constrained to zero in pp whilst it was taking large values in the optimisation as it was unconstrained. No idea whether this had a significant impact or not but you may want to check once you've aligned both

julie-vienne commented 1 year ago

I am assuming this error occurs when learning from optimisations right? no in pure exploration-based learning. What is the gap? Is it lower than the tol_cons_constraints?

I've merged again with main and was not able to reproduce the error. It seemed all fixed with your latest commits. Thanks and sorry!

julie-vienne commented 1 year ago

Also this means that all along we were not comparing the same thing when comparing the optimisation results and the pp simulations, as reactive power was constrained to zero in pp whilst it was taking large values in the optimisation as it was unconstrained. No idea whether this had a significant impact or not but you may want to check once you've aligned both.

Yes absolutely! I guess that since our losses were anyway very low, the reactive power did not have a significant impact on the voltages or any other variables but it's great that we've caught it now!

julie-vienne commented 1 year ago

@floracharbo you can have a look at #29 to see how it's been implemented. I'm testing it right now to see if it leads to any issues.

@floracharbo see commit 9329a63705e75182a0326a54a9d4bc31bab886f5. The reactive power seems to work with 0 passive homes and not yet q_battery as a decision variable. I will close this if you don't see any more problems here.