search

ibpsa / project1-boptest

Building Optimization Performance Tests
Other
104 stars 69 forks source link

Unit tests depends on list order that is not determintistic #529

Closed kbenne closed 1 year ago

kbenne commented 1 year ago

This line and the very next one depends on a determinisitic order of object keys

https://github.com/ibpsa/project1-boptest/blob/b040c4283476ba3e7db76d70d43c789946ff3627/testing/utilities.py#L604

While testing BOPTEST-Service I discoverd this test failing due to the nondeterminism. (Service appears to be sending the keys back in the oposite order) I think the test should not assume a particular order.

https://stackoverflow.com/questions/42232200/as3-deterministic-json-string-from-an-object

@dhblum

dhblum commented 1 year ago

Thanks for pointing out @kbenne. I agree, and see it's because of forecast_points[0] in the next line 605. I suggest updating to specify the one point to fetch in the test setup for each test case as is already done for singular input and measurement checks, for example as in https://github.com/ibpsa/project1-boptest/blob/b040c4283476ba3e7db76d70d43c789946ff3627/testing/test_bestest_air.py#L54.

dhblum commented 1 year ago

Closed by https://github.com/ibpsa/project1-boptest/pull/530.