search

ibpsa / project1-boptest

Building Optimization Performance Tests
Other
104 stars 69 forks source link

Issue520 array append #521

Closed dhblum closed 1 year ago

dhblum commented 1 year ago

This is for #520. In particular, it changes the use of numpy arrays for storing y_store and u_store in testcase.py to using python arrays, which is more efficient in memory usage.

dhblum commented 1 year ago

Sorry, but in thinking about this more, I'm having second thoughts on moving to single precision storage. That is because Python naturally only really uses double precision (see the description under numbers.Real(float) here), except I guess for possible storage of single precision in python arrays. So, if I do something like:

>>> L = array.array('f', [275.15])
>>> L[0]
275.1499938964844

whereas

>>> L = array.array('d', [275.15])
>>> L[0]
275.15

But BOPTEST uses Python operations on that data, including KPI calculation and returning the data to users when asked for. KPI calculation I'm not as worried about, but the former is what would be returned to a user using the API request /results in the case of storing as single precision. So, for example, if a user specifies a control value to be 275.15 and then asks for the history of value(s) later in the simulation, they will get 275.1499938964844, and likely question what's going on. Similarly, if a model variable is initialized to a value of 275.15, and a query is made to a corresponding sensor, the user will get 275.1499938964844. Therefore, I'm tempted to implement the python arrays in this PR but keep storing as "d" until more thought can go into properly representing to users and if there's a need for memory reduction that is solved by "f" storage.

EDIT: Note this behavior is seen in the unit test result diffs which query for results.

kbenne commented 1 year ago

Sorry, but in thinking about this more, I'm having second thoughts on moving to single precision storage. That is because Python naturally only really uses double precision (see the description under numbers.Real(float) here), except I guess for possible storage of single precision in python arrays. So, if I do something like:

>>> L = array.array('f', [275.15])
>>> L[0]
275.1499938964844

whereas

>>> L = array.array('d', [275.15])
>>> L[0]
275.15

But BOPTEST uses Python operations on that data, including KPI calculation and returning the data to users when asked for. KPI calculation I'm not as worried about, but the former is what would be returned to a user using the API request /results in the case of storing as single precision. So, for example, if a user specifies a control value to be 275.15 and then asks for the history of value(s) later in the simulation, they will get 275.1499938964844, and likely question what's going on. Similarly, if a model variable is initialized to a value of 275.15, and a query is made to a corresponding sensor, the user will get 275.1499938964844. Therefore, I'm tempted to implement the python arrays in this PR but keep storing as "d" until more thought can go into properly representing to users and if there's a need for memory reduction that is solved by "f" storage.

EDIT: Note this behavior is seen in the unit test result diffs which query for results.

I support either direction that you want to go.

dhblum commented 1 year ago

Closing in favor of https://github.com/ibpsa/project1-boptest/pull/522.