EMS actuator override of mass flow rate and temperature provide strange behaviour.

[LorenzoBottaccioli]

Dear,

I'm developing new Reinforcement Learning Control Strategies and I'm using E+ as a test bed for the algorithms. I managed to use only

EnergyManagementSystem:Actuator,
    t_in_air,                !- Name
    VAV SYS 1 OUTLET NODE,   !- Actuated Component Unique Name
    System Node Setpoint,    !- Actuated Component Type
    Temperature Setpoint;    !- Actuated Component Control Type

as an actuator to optimize the Energy Consumption in the building by varying the temperature. Since I will like to give more control actions I have started to use the mass flow rate and I have created the following actuator:

EnergyManagementSystem:Actuator,
    mdot_fan,                !- Name
    SUPPLY FAN 1,            !- Actuated Component Unique Name
    Fan,                     !- Actuated Component Type
    Fan Air mass Flow Rate;  !- Actuated Component Control Type

Using the ExternalInterface:Variable, EnergyManagementSystem:ProgramCallingManager and EnergyManagementSystem:ProgramI pass and set my control actions:

EnergyManagementSystem:ProgramCallingManager,
    act_calls1,              !- Name
    InsideHVACSystemIterationLoop,  !- EnergyPlus Model Calling Point
    Act;                     !- Program Name 1

EnergyManagementSystem:Program,
   Act,                     !- Name
   SET t_in_air = tair,     !- Program Line 1
   SET mdot_fan = mdot;     !- Program Line 2

ExternalInterface:Variable,
    mdot,                    !- Name
    1;                       !- Initial Value

ExternalInterface:Variable,
    tair,                    !- Name
    1;                       !- Initial Value

I have tried with both AirTerminal:SingleDuct:ConstantVolume:NoReheat and AirTerminal:SingleDuct:VAV:Reheat.

With AirTerminal:SingleDuct:ConstantVolume:NoReheat the RL algorithm discovers a strange behaviour of E+ that makes the temperature to follow the E+ base line in blue but the HVAC consumption is almost 0

With AirTerminal:SingleDuct:VAV:Reheat the VAV SYS 1 OUTLET NODE it does not flow the controls that I pass to E+ (in green what E+ does in red what I wanted to do).

At the links you find the zips with the IDF, the control_actions and the E+ output for both cases.

https://www.dropbox.com/s/ektupyzrxaqbue7/VAV_costantVolume.zip?dl=0

https://www.dropbox.com/s/ev9ofl1wptpx0i5/VAV_reheat.zip?dl=0

Details

Some additional details for this issue (if relevant):

• Platform Ubuntu 18.04
• Version of EnergyPlus 9.6
• Unmethours link or helpdesk ticket number https://unmethours.com/question/70481/ems-actuator-override-of-mass-flow-rate-and-temperature-provide-strange-behaviour/

[bonnema]

Have you looked at the .edd file?

[LorenzoBottaccioli]

@bonnema yes I did. What shall I look for? This is the IDD file for the ConstantVolume idf eplusout_edd.zip

[bonnema]

I am having trouble following all that is going on, but the .edd files shows line-by-line execution of the EMS code. Does each line match what you are wanting it to do? For a given timestep and the given inputs, are the outputs being calculated correctly? Then is there a line setting the actuator to that output value? Those are things to check for initially.

You also have a lot of programs with different calling points, there may be some syncing issues. You could try to condense it all into a single program at the inside HVAC iteration loop calling point. That will make debugging easier.

[LorenzoBottaccioli]

@bonnema I'll try do what you suggested.

However, the big problem is that in Constant Volume the buildings heat up the internal temperature with no HVAC Power consumption! How is this possible?

[bonnema]

What's going on outside? Internal gains?

[LorenzoBottaccioli]

Wrt the Blueline E+ baseline simulation, the only difference are the controls of the Supply Air Temperature and the Supply Mass Flow Rate.

[bonnema]

It's really hard to say since you're actuating HVAC components with EMS.

[LorenzoBottaccioli]

@bonnema Is there a way to control both Supply temp and Mass flow rate? I don't understand why E+ is providing this behaviour.

[LorenzoBottaccioli]

I post here another example of what is happening without the external interfaces but just using actuators of the EMS.

No_Action means E+ running with his own logic. Action means setting 0 to mass flow rate and supply temperature equal to the mixed air temperature. Results the building gets hotter with 0 mass flow rate and supply temperature equal to the mixed air temperature.

idf.zip

Attached you find the IDF files.

[bonnema]

@LorenzoBottaccioli, can you explain more what you're trying to do? I took a look at your IDF. You EMS code seems to want to set the mass flow to zero and the temperature setpoint to the mixed air value. If you want to shut off the fan there is probably a better way (e.g., set the fan availability schedule to zero). Your temperature setpoint actuator is on the VAV Sys 1 Outlet Node. Your supply air temperature setpoint manager (SetpointManager:MultiZone:Heating:Average) is on the Supply Air Temp Nodes 1 node list, which only contains the VAV Sys 1 Outlet Node. I suspect there is some "fighting" going on between the E+ setpoint manager and your actuator. You may want to try changing the supply air temperature setpoint manager to a scheduled type and then actuating that schedule value.

[LorenzoBottaccioli]

@bonnema I'm trying to control both the mass flow rate and Supply air temperature with a Reinforcement learning algorithm in order to optimize energy consumption. The case of the last two idf I uploaded was just to stress the issue of the strange behaviour I'm facing. What strategy do you suggest to fix such an issue? The objective is to control both supply air temperature and mass flow rate. Moreover, the final objective is the control of the overall supply air temperature and the single mass flow rate of each zone.

Ciao, Lorenzo

[bonnema]

What is the algorithm learning from?

[rraustad]

Given that there are 2 independent variables that combined yield unlimited variation in delivered capacity I would say your title for the issue is accurate. Since the equations used to determine how the external interface is calculating both supply mass flow rate and temperature were not provided, with what I would assume to be the expectation to hold, or closely hold a thermostat temperature, then it is hard to comment on the resulting thermal response.

[rraustad]

Looking at this figure E+ holds a constant flow with a varying supply air temp to maintain the zone thermostat temp. And this model does a good job of holding that same profile. At set back at night as the zone temp floats down, E+ has constant flow and gradually increases the HVAC power to provide enough heat to approach and hold the Tstat temp. This model minimizes air flow when the zone temp hits a minimum and that allows the zone temp to float up either because of stored heat or less OA if provided. If the zone temp is above the set back temperature then no heating is required. E+ applies heat gradually to approach the set back temp and hold the Tstat set point. The difference here is that this model relaxes the set point so no heating is required. That is not a bad thing since the zone temp is actually more comfortable even if no one is there. This figure actually looks promising regarding the controls. In this figure on the 2nd day the zone temp floated up during the day with this model (because the HVAC was turned off?) and E+ targeted the temp set point. So here it looks like the target zone temp differs from what E+ is doing. Why there is no HVAC here is likely due to the different control of zone temp, E+ holds temp and this model allows the zone to float. For heating that's OK but the 2nd day shows there might have been the need for cooling? Maybe because this is a heating season and cooling is turned off? Add cooling here and control between a dual set point? Now if you want this model's zone temp to more resemble E+ then the HVAC system will have to turn on in cooling mode which may be provided just through fan operation if OA is included.

[LorenzoBottaccioli]

@rraustad this figure was just an extreme example of an anomaly I'm facing. In that case, there was no learning algorithm. The figure shows the same IDF file with the same weather conditions and the days refer to a winter situation in the first days of January. The difference is only that in the action case I have voluntarily set the inlet Temperature quale to mixed air temp and the mass flow rate at 0. Meanwhile, in the other case, E+ is running with its own logic. I don't understand why in the action case the room heats up with a lower inlet temperature and no flow rate and 0 consumption.

[LorenzoBottaccioli]

What is the algorithm learning from?

The algorithm is learning from E+

[rraustad]

@LorenzoBottaccioli if you turn off the HVAC system (0 mass flow) then the zone will float to whatever temperature based on the total zone heat gain. I don't know what any of the simulation inputs are, lighting, equipment, etc., or if there is outdoor air when HVAC system in on. Is your model trying to maintain a zone temperature? I suspect it is but not sure what controls the HVAC system to turn on and off (i.e., how supply mass flow rate and supply temperature are controlled).

[bonnema]

What is the algorithm learning from?

The algorithm is learning from E+

I am not sure how well learning from a model will go. I think you could solve for this at each timestep. Similar to what @rraustad said, you have to meet the load with a flow rate and a supply temperature. You probably have an equation that defines fan power (typically as a function of flow rate) and an equation that defines cooling power (typically as a function of both indoor and outdoor conditions). There should be a value for each that together result in the lowest energy.

[mjwitte]

I think this discussion has veered off track. The initial question is whether the actuator is being applied correctly. @LorenzoBottaccioli Looking at the detailed node conditions should help diagnose what is happening. Be sure to report at the "detailed" reporting frequency so there is not averaging. And adding some custom EMS output variables that reflect the actuated values for mass flow rate and temperature setpoint would be useful too. (My apologies if this is already present in the outputs. I haven't had a change to look at the idf files yet.)

[vipsam1987]

Hello Lorenzo,

Any luck with the issue? Have you been able to successfully control the mass flow rate and supply temperature into each zone?

Thank you. Heman

[Sdrd01]

Hello,

I have the same question as @vipsam1987 . I'm trying to control the temperature and air flow in each zone of my system, but I can't find anything convincing...

Is there a solution?

Thank you and have a nice day.

[mjwitte]

@Sdrd01 Looking at the initial request . . .

1. "With AirTerminal:SingleDuct:VAV:Reheat the VAV SYS 1 OUTLET NODE it does not flow the controls that I pass to E+ ". That is expected. The VAV terminal unit will attempt to control the airflow rate, and the actuator on the fan mass flow rate will fight with this.
2. "With AirTerminal:SingleDuct:ConstantVolume:NoReheat the RL algorithm discovers a strange behaviour of E+ that makes the temperature to follow the E+ base line in blue but the HVAC consumption is almost 0". Looking at the attached idf, there are multiple mass flow rate actuators, one for each zone terminal unit:

   
   EnergyManagementSystem:Actuator,
   mdot_1,                  !- Name
   SPACE1-1 VAV NOREHEAT,   !- Actuated Component Unique Name
   AirTerminal:SingleDuct:ConstantVolume:NoReheat,  !- Actuated Component Type
   Mass Flow Rate;          !- Actuated Component Control Type

etc.

Plus one for the supply fan and the mixed air node.

EnergyManagementSystem:Actuator, mdot_fan, !- Name SUPPLY FAN 1, !- Actuated Component Unique Name Fan, !- Actuated Component Type Fan Air mass Flow Rate; !- Actuated Component Control Type

EnergyManagementSystem:Actuator, mdot_air, !- Name MIXED AIR NODE 1, !- Actuated Component Unique Name System Node Setpoint, !- Actuated Component Type Mass Flow Rate Setpoint; !- Actuated Component Control Type

If the terminal units are controlled, then the supply fan should follow that without the need for an actuator on the fan. A mass flow rate setpoint on the mixed air node will do nothing. Mass flow rate setpoints in general do nothing in EnergyPlus.

So, I took the 5Zone_costantV_Control_T_m.idf file from above and modified it to use only EMS (no external interface) and set the zone mass flow rates to 0.25, the fan mass flow rate to the sum of the zone mass flow rates, and the supply air setpoint to 20.0 just to see if the actuators work as expected. And added more output variables, all at the "detailed" reporting frequency.

1. There was no cooling coil operation, because the CoolingCoilAvailSched was always zero. Changed that to always 1.0.
2. The mass flow rates were not as expected, because 0.25 was greater than the autosized max flow rate for some of the terminal units. Changed the EMS to set all zoned to 0.20 kg/s. Now the mass flow rates are all 0.20 as expected, and the fan is 5x0.20=1.0.
3. The supply air setpoint actuator is being called at InsideHVACSystemIterationLoop. This is overriding the setpoint at VAV SYS 1 OUTLET NODE as expected, but that does not propagate to the upstream nodes at the cooling coil and heating coil, because the regular setpoint managers are executed before the InsideHVACSystemIterationLoop calling point. So, the EMS program needs to set the adjusted setpoint at the cooling coil outlet (as in example file EMSReplaceTraditionalManagers_LargeOffice.idf), or change the DX Cooling Coil System Sensor Node Name in CoilSystem:Cooling:DX to VAV SYS 1 OUTLET NODE. This works almost perfectly, the supply air temp is always very near 20.0C, but there is a little bit of simultaneous heating and cooling, because using a downstream control node can introduce some setpoint error. It's better to set the cooling coil and heating coil outlet node setpoints directly (adjusted for fan heat).

So, in conclusion. These actuators appear to be working correctly, but there are many other considerations to conquer in order to get the expected results.

[mjwitte]

Here's the final idf. 5Zone_costantV_Control_T_m-EMSOnly2.idf.txt