overthesun / simoc

A scalable model of an interactive, off-world community
https://ngs.simoc.space/
GNU General Public License v3.0
2 stars 2 forks source link

Add a basic testing framework #129

Open ezio-melotti opened 3 years ago

ezio-melotti commented 3 years ago

A test framework and a basic test suite to validate the results produced by the backend should be added.

In its simplest form it can be a standalone script that requests all the steps to the backend and compares them with the expected result. We can integrate it with the backend, call the functions directly, and access the results from Redis too, but that might be a bit more involved.

While testing the results might be a bit off due to the random evaluation order of the agents and rounding, so this should be taken into account (e.g. with fuzzy matching). Disabling random evaluation and using a fixed seed should also fix this issue.

kstaats commented 3 years ago

Thank you.

On 6/6/21 3:55 PM, Ezio Melotti wrote:

A test framework and a basic test suite to validate the results produced by the backend should be added.

In its simplest form it can be a standalone script that requests all the steps to the backend and compares them with the expected result. We can integrate it with the backend, call the functions directly, and access the results from Redis too, but that might be a bit more involved.

While testing the results might be a bit off due to the random evaluation order of the agents and rounding, so this should be taken into account (e.g. with fuzzy matching). Disabling random evaluation and using a fixed seed should also fix this issue.