ibpsa / project1-boptest

Building Optimization Performance Tests
Other
109 stars 70 forks source link

Measurement accuracy/resolution #483

Open haraldwalnum opened 2 years ago

haraldwalnum commented 2 years ago

We previously discussed implementation of measurement noise in the emulator model, and the implications on simulation time. But I thought a first step could be to reduce the resolution of the measurements returned through the API.

Currently, all measurements are returned as double precision floating point, meaning 16 digits. This is a resolution that you will never get from a BMS, and if you do, you cannot trust it. My guess is that this could have significant impact on controller performance, especially when controlling based on slow measurement points (e.g. indoor temperature, in combination with floor heating) or with short control steps.

My first thought was that this fairly simple to be implemented in __get_results One challenge is that different measurement types could require different resolution (or decimal points). For many cases (e.g. temperature) 1 decimal point would probably make sense. For values like valve position it is maybe not enough.

dhblum commented 2 years ago

Thanks for bringing this up. I agree this could be an issue to address to make the values be more realistic and also, from a computational efficiency standpoint, probably save some memory and API bandwidth to not have to report so many numbers. I'm curious to hear more about what you're thinking in terms of significant impacts on controller performance though?

One option is to have a new parameter to the signal exchange block where a developer can specify a resolution for the measurement point and add it to the metadata that gets stored with each point, which can then be read by __get_results upon request or even stored that way (although the latter would require looping through all the points each control step and cutting off digits before storing, which could be expensive in the long run).

haraldwalnum commented 2 years ago

I agree that adding a parameter to the signal exchange block is probably the best solution. Even though it would require modifications to all the emulators.

For saving memory, I don't think rounding off floats will save memory as long as they are stored as floats.

Regarding effect on controller performance, it was something that came to my mind, but I had not tested. I have now done a quick test on the Bestest_hydronic_heat_pump testcase, just rounding off the indoor temperature measurement, and found no significant impact on the cost KPI, even when rounding of to 0 decimal points. On tdis_tot, I got about 3 % poorer performance. I am not sure this fully disproves the hypothesis though.

haraldwalnum commented 2 years ago

I have done some initial testing with the solution where the round function is implemented in the read block, so that y is rounded to the desired precision before read by BOPTEST or used by the baseline controller. As discussed in the meeting.

To test if the implication on the simulation, I tested to add a round block (Buildings.Controls.OBC.CDL.Continuous.Round) to the PIDHysteresis example in Buildings (Buildings.Controls.Continuous.Examples.PIDHysteresis), between the temperature sensor and the PID controller.

image

The results were not promising. Without the round function, the model solved (1 day) in 0.007s, while with round(n=1), it just stalls at T=4228s. Setting n=5 it solved in 41s with CVODE, but increasing n further increases the simulation time again. I tried with some different combinations of solvers and tolerance, but with similar results.

Replacing the PIDHysteresis controller with a pure Hysteresis controller, it solved in 0.037 seconds with n=1 and 230s with n=5.

Based on this, it seems that the round function has significant influence on simulation time, especially when the output signal is used in a continuous controller. The latter making it a non-feasible solution.

Based on this I think that the only feasible way forward is to implement a parameter in the read block that specifies the precision, but to the rounding in _get_results. The disadvantage is that the rounding does not influence the baseline controller, and external high level controllers that utilize the internal controllers (e.g. setpoint controllers) would benefit, compared to more low level controllers. The benefit is that it is easy to turn on/off the rounding function through the BOPTEST API, so that the rounding function is only applied in specific scenarios.