Implement a System for Backwards Compatible Output Variable Names with Clear Deprecation Warnings

chriswmackey commented 3 weeks ago

Issue overview

I am opening this issue after struggling with a trend that has been happening in the last few stable releases of EnergyPlus - the names of some output variables keep changing. Granted, I fully understand the need to revise the output names but this trend has been breaking many workflows within the Ladybug Tools software that I maintain, which is used by tens of thousands of energy modelers. I have also heard many other EnergyPlus users outside of Ladybug Tools struggling with these changes, including:

I recall some members of the OpenStudio team mentioning that they had to delay a stable release because they had to update the variable names in hundreds of measures.
A couple of consulting offices have mentioned to me that they are currently not upgrading their version of EnergyPlus in-part because they have a collection of measures that similarly require updating of output variable names.
This last one is a little speculative but I know the version of E+ used by the latest DesignBuilder software is from 4 years ago. I imagine their reasons for not updating are multi-faceted but having to identify and change many output variable names clearly makes their job a lot harder.

As a software developer who maintains an interface to E+, I find that the worst part of these output name changes is that they tend to go unnoticed until someone on the Ladybug Tools forum asks why their results do not make sense and it's simply because the output variable name changed and so it was not included in the results. So I have personally oversaw several buggy releases of Ladybug Tools because it was initially not obvious to me that an output name had changed.

Granted, I know that E+ currently gives the following warning:

   ** Warning ** The following Report Variables were requested but not generated -- check.rdd file
   **   ~~~   ** Either the IDF did not contain these elements, the variable name is misspelled,
   **   ~~~   ** or the requested variable is an advanced output which requires Output : Diagnostics, DisplayAdvancedReportVariables;

But this tends to get lost in the sea of other minor warnings for large models and it's always really hard for me to pick out the output variables in the list that have actually changed names vs. ones that I have added to the IDF in case they become relevant in a future version of the model.

Proposed solution

As I was discussing this with @MingboPeng, he suggested a relatively simple but potentially very helpful idea. What if E+ kept an internal record of output variable names that have changed in recent releases and, before appending the unrecognized output to the good old list of "following Report Variables were requested but not generated", E+ checks whether the unrecognized output variable is a previously-supported output name.

This way, E+ can give a ** Severe ** deprecation warning that the output variable name has changed, which is something that I will notice during my testing before I perform a stable release of Ladybug Tools. This would be so much better than hiding it within a minor warning with other variables and it would go a very long way to making sure that I don't release buggy versions of my software.

Furthermore, what would really make my day (and probably help a lot of others update to the latest E+ version) is if, when E+ identified an output variable that was historically used, it re-routes it to request the correct/updated output variable. This way, I don't get to the end of an hour waiting for an E+ simulation to finish only to realize that it does not have the results that I thought I requested. This would also help those offices that are not updating their E+ to just get something that works in the latest E+ version for now and then they can fix all of the output variable names later by going through the severe warnings.

Granted, I'm not married to the exact proposal above but it's clear to me and a lot of other users of E+ that something really needs to be implemented to better handle the name changes of output variables. The current system just results in many mistakes and lots of wasted human-hours. There has to be a better way to do this.

Checklist

Add to this list or remove from it as applicable. This is a simple templated set of guidelines.

[ ] Ticket added to EnergyPlus Defect Complexity (Github Project)
[ ] Pull request created (the pull request will have additional tasks related to reviewing changes that fix this defect)

Vbraciszewski commented 3 weeks ago

Thank you for submitting this issue Chris. Hopefully, it gets the attention it deserves from NREL team.

rraustad commented 3 weeks ago

There is a list of the changes in ...\PreProcess\IDFVersionUpdater\Report Variables*.csv

chriswmackey commented 3 weeks ago

Thanks, @rraustad. Those CSVs are really helpful but I still can't describe what a world of difference it would make to have the data in these CSVs incorporated into E+ so that I could get Severe warnings about them. Particularly when we have a change like what happened between E+ 9.3 and 9.4, where over 540 outputs changed names, I'm almost certainly going to miss some as I try to update our software:

NREL / EnergyPlus