Normalize energy usage by floor area

[dhblum]

This issue is to address the normalization of energy usage by floor area of the test case, which could make it possible to compare results with other case studies on similar buildings, or potentially across building types. Though, different building types will naturally have different EUIs within a likely range.

[javiarrobas]

I like this initiative. Is the idea to compute the EUI as an additional core KPI? or substituting of the currently used ener_tot? or just as another KPI not belonging to the core set of KPIs?

[mwetter]

I would rather substitute the energy consumption (kWh) with normalized energy consumption (kWh/m2). You could separately track the floor area so you can see which buildings are within a certain size, but comparing kWh across buildings with different floor area is not meaningful. This is why I think we should be using kWh/m2.

[dhblum]

@mwetter I wanted to open this discussion again to point out that our test scenarios now are not annual tests, but two-week periods at the moment. Perhaps this gets expanded in the future. But my notion of kWh/m2 is that it is usually associated with annual energy metrics and I have a bit of concern that reporting normalized by floor area may add some confusion when the numbers are not comparable to EUIs for similar building types found elsewhere, without further understanding that they are not meant for annual EUI comparison. Perhaps this is of low concern, but wanted to see what others think.

I also want to raise the question of if cost, discomfort, and CO2 emission core kpis should also be normalized by floor area?

[mwetter]

@dhblum : I still think it makes sense to normalize by the floor area to remove the dependency on the arbitrary size of the building. One way to avoid confusion with EUI would be to report kWh/(m2.a) for annual, as it is done in some energy codes, and maybe kWh/m2 otherwise. Or if one has multiple 2 weeks periods in different seasons, maybe a weighted sum could be used to extrapolate from short-term measurements to expected annual performance, but this would need careful design and/or research.

[javiarrobas]

@mwetter is your idea to normalize by the floor area only the energy use? or are you thinking of normalizing also others like operational cost, emissions, thermal discomfort, IAQ, etc. ?

[mwetter]

@JavierArroyoBastida I suggest to normalize everything that scales with floor area, such as cost and emission. However, thermal comfort should be an average (or max discomfort) across all zones, maybe weighted by floor area and/or importance so that a large office zone may get more weight than a small hallway.

One way is to think about is that the performance should be scale invariant: An MPC controller should have the same performance if it controls one building or two identical buildings with the same disturbances.

[dhblum]

Thanks @mwetter. I think normalizing energy, cost, emissions by floor area is ok. Extrapolating short-term test to annual results is indeed a research project on it's own being undertaken by PNNL, which @SenHuang19 could provide more details for, and is not yet ready for BOPTEST implementation from my impression. Thermal comfort (K-h) could simply be weighted by total floor area, which would represent an average of all zones. Weighting the thermal comfort by zone is also a bit of a study I think, considering if you replace in your example the small hallway with a small CEO's office and large office with a large common space (like a lobby or cafeteria). Then, weighting by area may not suffice, and weighting factors need to be developed for space types.

[javiarrobas]

Actually, I think that thermal discomfort (and other equivalent KPIs like IAQ) should be normalized by the number of conditioned zones, or more precisely, by the number of read blocks that are used to calculate that KPI. The KPI would be in units of Kh/zone as an average of all zones, independently of their size. To understand why I believe this is more meaningful, think of the bestest_hydronic vs. the multi_zone_residential_hydronic case. The former has a total area of 192m2 and only one conditioned zone. The latter has a total area of 120m2, and six conditioned zones. If normalization is meant for an MPC to deliver similar performance even when tested for two different buidlings, then normalizing by floor area will not help in this particular case. The reason is that, obviously, the multi-zone case is expected to generally show higher thermal discomfort only because there are six potential sources of thermal discomfort, one for each indoor temperature reading. Dividing by total floor area only gets things worst since the area of the multi-zone case happens to be smaller than the single-zone case.

[yanchenpnnl]

@dhblum If some of the test cases are not simulating for the whole year, it would be hard to compare the absolute values. In those cases, would it make more sense to emphasize more on using the relative (saving%, impact%) of the MPC controller, comparing to the baseline case? For example, in the building energy code analysis, the relative savings are often used because even if a software update creates absolute EUI changes, the saving percentage between baseline and advanced case typically stays similar. I also agree to normalize the energy, cost, emission to the conditioned space area. Potentially, the IAQ comfort can also be normalized as the percentage of the whole simulation period, and for a multizone system, it can use zone area /conditioned area to weight/aggregate the whole building values.

[dhblum]

@JavierArroyoBastida I think you may have a point with the fact that if you take 6 measurements vs. 1 you have 6x the opportunity for thermal discomfort. But just to clarify Michael said "should have the same performance if it controls one building or two identical buildings with the same disturbances." I should also point out I made a mistake in my last comment. Summing the thermal discomfort of each zone and then dividing by total floor area is NOT the same as summing over all zones the ratio of thermal discomfort to zone floor area.

@yanchenpnnl At first I wasn't sure why one would compare the KPIs between two test cases, since that would be apples to oranges. But I think I see if someone wanted to see easily across test cases if a particular controller always performed at least 20% better than baseline. The only thing is, another important aspect is to compare MPC1 vs. MPC2, so the baseline controller is not always the %savings you'd want to see. Also, if a user asks for a KPI in the middle of a test, as RL controllers would or as Javi has done to show when exactly MPC particularly outperforms Baseline, would it be confusing to report that value as %savings over a baseline controller that has run the entire test scenario? And what if a user asks for KPIs after running a simulation in a period outside the specified test scenario, say as practice or for model training? There will be no already-established-baseline values for that time period specified by the user?

[dhblum]

One more comment on the %savings, my inclination is to provide consistent fundamental information without too many assumptions built in, which can be left to a user (or additional processing module) to decide how to best get the answers they want to get for their specific questions.

[dhblum]

On the topic of thermal discomfort, I have not found a reference that suggests weighting thermal comfort by zone area. I have found the following:

• ASHRAE Standard 55 states in so many ways that thermal comfort should be maintained and evaluated for each conditioned space.
• Oldewurtel et al. 2012 considered only a single zone in the study and reported K-h.
• Sourbron et al. 2013 considered two zones and reported total K-h.
• DeConinck and Helsen 2016 considered multizone and averaged zone temperature measurements to compute minutes uncomfortable (< 21 C).
• Sturzenegger et al. 2016 considered multizone and reported the max K-h over all zones.
• Drgona et al. 2020 considered multizone and reported average K-h per zone per day.

I do not think it makes sense to weight by floor area since that would incentivize concentrating on large zones, which isn't the goal in practical implementations, particularly if you have a building with a large common area and small private offices. Taking the average discomfort over all zones (which require heating/cooling) could lead to ignoring single zones that remain very uncomfortable. Therefore, I agree most with the method of Sturzenegger et al. 2016 who reported the max K-h over all zones.

One further improvement to this could be to integrate using the maximum deviation over all zones at any time, rather than integrating the deviation for each zone and then taking the maximum. This metric would account for the possibility that, for instance, zone 1 is uncomfortable during one time period and not during a second and zone 2 is uncomfortable during the second time period and not during the first. A controller that lets this happen would be penalized "twice," rather than effectively "once" in the Sturzenegger implementation. I think this would also have the same evaluation effect as summing all discomfort for all zones as we do now, but make the magnitude invariant to the number of zones.

EDIT: This could apply to IAQ [ppm-h] as well. EDIT: Added Sourbron reference.

[javiarrobas]

I think what you propose @dhblum makes sense and agree with integrating using the maximum deviation over all zones at any time.

[dhblum]

Thanks @JavierArroyoBastida, but sorry, another thought here. The proposed approach of integrating the maximum deviation over all zones will yield the same discomfort result for two controllers where one allows zone 1 to have small discomfort and zone 2 to have large discomfort and the other allows zone 1 to have no discomfort and zone 2 to have large discomfort. I think in this case, the KPI should indicate the first controller performs worse, but it won't.

I'm starting to think averaging discomfort over all zones is the best way. I think it will represent discomfort in each zone just as well as the simple sum of discomfort over all zones (contrary to what I said before), but invariant to the number of zones.

[javiarrobas]

Good catch. I didn't think of that scenario, but indeed, KPIs should reflect that the first controller performs worst. Hence, if weighting by floor area is not common practice in practical implementations, I also agree that averaging over all zones is probably what makes most sense. I like of this one that is easy to implement and to understand.

[dhblum]

I'd like to make a proposal in order to move forward with a necessary PR. I propose the KPIs be adjusted to:

• Energy, Cost, and Emissions be normalized by total building floor area such that units are kWh/m^2, $ or Euro/m^2, and kgCO2/m^2.
• Thermal Discomfort and IAQ Discomfort be normalized by the total number of conditioned zones such that units are K-h/zone and ppm-h/zone.

Please indicate if this is unreasonable.

[javiarrobas]

Thanks @dhblum for the proposal. I think it makes sense and I agree with it.

[mwetter]

@dhblum : These are fine for me. Note however it should be K.h/zone rather than with a minus sign, see also https://physics.nist.gov/cuu/pdf/sp811.pdf

[dhblum]

Thanks @JavierArroyoBastida and @mwetter. Noted about the use of a period rather than minus sign, and thanks for that reference.

[Mathadon]

On the discomfort quantification issue. I completely agree with this:

@JavierArroyoBastida I suggest to normalize everything that scales with floor area, such as cost and emission. However, thermal comfort should be an average (or max discomfort) across all zones, maybe weighted by floor area and/or importance so that a large office zone may get more weight than a small hallway.

One way is to think about is that the performance should be scale invariant: An MPC controller should have the same performance if it controls one building or two identical buildings with the same disturbances.

And I don't really agree with this:

I do not think it makes sense to weight by floor area since that would incentivize concentrating on large zones, which isn't the goal in practical implementations, particularly if you have a building with a large common area and small private offices.

I would turn this argument around: If you don't weight by floor area, the controller may heat only small zones and accept a (relatively minor) 'penalty' in the KPIs for completely neglecting the larger zone, while obtaining massive energy savings due to that.

We have to consider here that smaller zones require less energy to be heated/cooled compared to larger zones so while the benefit (comfort KPI) of heating the zone will be less incentivizing to actually heat the zone, the cost for doing so is also less. The end result will/should be that the zone is heated even if it has a smaller scaling factor.

I would therefore area-weight the discomfort penalty and sum all penalties.

[dhblum]

I do not think we can assume smaller zones require less energy and that the cost of keeping smaller zones comfortable is always less than larger zones. Consider a smaller zone on the perimeter of the building dominated by envelope loads against a larger zone on the interior of a building with no envelope loads. Or consider a smaller zone with high occupant/equipment density against a larger zone with low occupant/equipment density.

[Mathadon]

You're comparing apples with pears there :p

[javiarrobas]

I've never seen the objective of an optimal controller to weight by floor area, even for multi-zone building cases. All slack variables associated to discomfort normally use the same weight in the objective (e.g. one order of magnitude larger than the maximum electrical power use) regardless of the area of each zone. This indicates that controller developers value the same each contribution of discomfort, independently of the nature and size of the conditioned zone where it comes from.

[Mathadon]

That analysis is correct but I don't agree with the rationale.

As Michael mentioned, the optimisation problem should be scale invariant. E.g. suppose you have a building model and you decide to split a zone into two zones. The resulting models both represent the same physical system and thus ideally would achieve the same optimal solution. If you don't weight the surface area, then there will now be two comfort KPIs pushing down discomfort in the same physical room instead of 1. If you weight the KPI then you can get to the same solution (depending on how you split the zone exactly of course).

A different take on it: suppose that two controllers achieve identical KPIs but 1 controller has discomfort in a large zone, where 50 people are present and 1 controller has discomfort in small zone where only 1 person is present. I think that the discomfort for 1 person is less problematic than discomfort for 50 people so that should be factored in. A good proxy for the number of occupants is the surface area.. Comfort is about people and if all people are equal then their comfort perception should be added up, imo.

[dhblum]

You're comparing apples with pears there :p

How so?

I am saying that I disagree with the argument that smaller zones always require less energy. I also disagree that a good proxy for the number of occupants in a zone is zone area alone. The occupant density of an office can be 5 ppl/100m^2, that of a conference room can be 50 ppl/100m^2, and that of a lecture hall 150 ppl/100m^2. See ASHRAE Standard 62.1 Table 6-1 on recommended default occupant densities for different space types.

I agree with the notion that 50 people comfortable is better than 1 person comfortable. Instead of using zone area as a proxy for number of people as a weighting, perhaps a proposal is use occupant number directly to weight, since we do have that information in simulation? See for instance the proposed "Exceedance_M" metric in https://www.tandfonline.com/doi/full/10.1080/09613218.2011.556345. It provides the fraction of occupied-hours satisfied weighted by number of occupants in a particular zone. They, and https://www.sciencedirect.com/science/article/pii/S0378778812003027, suggest it may be improved by using the temperature deviation from set point rather than a binary 0-1 to indicate acceptability, in order to additionally weight by more extreme deviations. Then, I think the proposed metric would be:

[Mathadon]

You're comparing apples with pears there :p

How so?

Since large/small zones would typically have similar heat loads per square meter and assuming otherwise is not the best assumption imo.

I am saying that I disagree with the argument that smaller zones always require less energy. I also disagree that a good proxy for the number of occupants in a zone is zone area alone. The occupant density of an office can be 5 ppl/100m^2, that of a conference room can be 50 ppl/100m^2, and that of a lecture hall 150 ppl/100m^2. See ASHRAE Standard 62.1 Table 6-1 on recommended default occupant densities for different space types. Sure, fair enough, but it's a better proxy than having no scaling factor at all.

I agree with the notion that 50 people comfortable is better than 1 person comfortable. Instead of using zone area as a proxy for number of people as a weighting, perhaps a proposal is use occupant number directly to weight, since we do have that information in simulation? See for instance the proposed "Exceedance_M" metric in https://www.tandfonline.com/doi/full/10.1080/09613218.2011.556345. It provides the fraction of occupied-hours satisfied weighted by number of occupants in a particular zone. They, and https://www.sciencedirect.com/science/article/pii/S0378778812003027, suggest it may be improved by using the temperature deviation from set point rather than a binary 0-1 to indicate acceptability, in order to additionally weight by more extreme deviations. Then, I think the proposed metric would be:

That indeed sounds even better than using the surface area. The question is then whether the controller developer has access to the number of occupants per zone. Imo they should not have that access since this is a measurement that is not typically available in practice.

We could drop the denominator since it is controller-independent.

[javiarrobas]

Thanks @dhblum for the references. Quoting a sentence of Comfort standards and variations in exceedancefor mixed-mode buildings: "definitions of comfort are not set in stone", and I think that's the problem we're facing here. The core KPIs were decided at the beginning or IBPSA Project 1 to be a trade-off between performance representativeness and complexity. I agree with normalizing by number of zones because it's clear that chances of discomfort are higher if there are multiple potential sources of discomfort, plus the resulting metric is still well understood. I'm afraid that if we go more complex (e.g. by including number of occupants in the metric) we can end up blurring the performance comparison of different controllers, which is the ultimate goal of BOPTEST. Even then, I'm sure we'll still get questioins like: why didn't you use PMV? Why didn't you follow other proposed metrics in standards ISO7730, EN15251, ASHRAE55 or ISSO74? BOPTEST also provides functionality to compute custom KPIs when needed, but comprehensibility should prevail in the in the core KPIs.

[dhblum]

You're comparing apples with pears there :p

How so?

Since large/small zones would typically have similar heat loads per square meter and assuming otherwise is not the best assumption imo.

Heat load per square meter depends on internal load density (occupant number + equipment + lighting, all dependent on space type) and relationship to exterior conditions. See here design W/m^2 for each zone in the prototype medium office building with ASHRAE 90.1 design in New York climate (See https://www.energycodes.gov/development/commercial/prototype_models). Note the difference between bottom and top vs. middle floor and core vs. perimeter. All zones here are of type office, so the variability added by internal load density isn't included.

The question is then whether the controller developer has access to the number of occupants per zone. Imo they should not have that access since this is a measurement that is not typically available in practice.

Yes, controller developers have access to deterministic schedules of occupant number. While they are not typically available in practice currently, there is a lot of work going on to enable economic, accurate measurement and prediction of the number of occupants in buildings for use in control. Making deterministic schedules available for use in BOPTEST enables it to be used to answer questions like, "What is the performance gap between controllers that use more or less accurate means (e.g. zone CO2 or equipment/lighting power as proxies) of predicting occupant numbers in a space?" If the gap is small, money and time can be spent developing other technologies or capabilities, and that is value that BOPTEST provides.

We could drop the denominator since it is controller-independent.

The original concept of this issue was to make KPIs scale-independent. The denominator would be needed to be scale-independent based on number of total occupants.

I agree with @JavierArroyoBastida that it is a challenge to reduce the complexity of controller evaluation down to only a few KPIs. And for thermal comfort especially since there are so many factors actually involved. Indeed we do, and will continue, to get questions on why these particular were chosen and why not others that have/will ever been used. I believe it needs to be based on capturing the important and common bases for evaluation as well as providing sound physical interpretation.

For discomfort, I am leaning towards taking the simple average of all zones. Reasons are:

1. It is easy (as I even did in my last comment) to theorize that it is objectively better to weight the KPI by number of occupants in the sense that "better to have more people comfortable than not" and each occupants weight towards achieving this objective is equal. But in thinking more, I actually am not sure this objectivity transfers to practice. The fact that a controller is ignoring a single or few occupants could be hidden from the evaluation and this could be inappropriate since a) each occupant deserves to be comfortable and b) it only takes a single occupant for the facility operator's phone to ring.
2. Splitting one large zone into two does provide an additional opportunity for at least one occupant to be uncomfortable, thereby changing the nature of the problem presented to the control system, and potentially the solution. If somehow this splitting would not also change each zone's load intensity, then the KPI is still scale-invariant by taking the average of the two zones. Whether or not an MPC controller solution is independent of such a change depends on the formulation of the specific MPC problem (to be independent, the formulation could minimize the average discomfort or treat comfort limits as hard constraints).
3. As I've argued, zone area is not a reliable (i.e. generalizable across all buildings) proxy for number of occupants nor heating/cooling energy. Only if the space types and environmental exposures are the same for the considered zones.
4. An additional consideration is the fact that the technical objective of current building control systems is to meet each zone set point or deadband, with the underlying assumption that the set point provided is comfortable for most occupants in the space at that time.

[Mathadon]

@dhblum you certainly make good points. I'm still in favour of an (area-)weighted objective but it's not really up to me decide what argument is most important in this discussion. Now that all the pro's and con's are on the table, feel free to pick the objective that you consider the best.

[dhblum]

Thanks @Mathadon. I appreciate the discussion to examine and document the different solutions and consequences. I suggest moving forward with what I proposed earlier and quote below:

• Energy, Cost, and Emissions be normalized by total building floor area such that units are kWh/m^2, $ or Euro/m^2, and kgCO2/m^2.
• Thermal Discomfort and IAQ Discomfort be normalized by the total number of conditioned zones such that units are K.h/zone and ppm.h/zone.

[dhblum]

I am suggesting here the definition of floor area so as not to be ambiguous (will be added to documentation as well).

For commercial buildings:

• We use the definition specified by CBECS as "Floorspace: All the area enclosed by the exterior walls of a building, both finished and unfinished, including indoor parking facilities, basements, hallways, lobbies, stairways, and elevator shafts."
• This is consistent with the definition of EUI given by EnergyStar as "EUI is expressed as energy per square foot per year. It’s calculated by dividing the total energy consumed by the building in one year (measured in kBtu or GJ) by the total gross floor area of the building (measured in square feet or square meters)."
• And is consistent with the definition specified by ASHRAE 90.1 as "Floor area, gross: the sum of the floor areas of the spaces within the building, including basements, mezzanine, and intermediate-floor tiers, and penthouses with a headroom height of 7.5 ft or greater [...] excluding covered walkways, open roofed-over areas, porches and similar spaces, pip trenches, exterior terraces or steps, chimneys, roof overhangs, and similar features."

For residential buildings:

• We use the definition specified by RECS as "In RECS, total square footage, or floorspace, is a measurement of the two-dimensional area of the housing unit that is enclosed from weather by exterior walls, which is also where residential energy-consuming activities occur. Total square footage can be broken down into four areas: attic, basement, garage, and the rest of the home. The basement and all living areas (the first floor, second floor, etc.) are always included in the RECS definition of square footage. Attics are only included if they are heated, cooled, or finished. Garages are only included if they are heated or cooled, and directly attached to the housing unit."

[yanchenpnnl]

Thanks @dhblum. This looks great, the building energy codes determination also follow the same floor area definition for reporting the energy performance related floor area.

[dhblum]

Thanks @yanchenpnnl that's great to hear! Thank you for the additional corroboration.

[dhblum]

Closed by https://github.com/ibpsa/project1-boptest/pull/330.