Automated Unit Testing

[BenjamenMeyer]

I see the following testing asset repositories:

• https://github.com/vegastrike/Testing-Win64
• https://github.com/vegastrike/Testing-Win32
• https://github.com/vegastrike/Testing-Mac

I don't see one for Linux though; I've been using https://github.com/vegastrike/Assets-Production for local play and testing on Linux.

Presently our CI system passes based on whether or not it can build the system; however, we need to add some functionality around unit testing and integration testing too. This will give us confidence in changes not breaking things and functionality working as expected.

Filing this to start the conversation and as an initial point for a larger Testing Project.

[BenjamenMeyer]

Could be wrong, but I don't think tests exist right now; so this is going to be a big effort to get this going.

Python Code can be tested using Python's Unit Test and Mock functionality. C/C++ code can be tested using https://www.boost.org/doc/libs/1_72_0/libs/test/doc/html/index.html

[stephengtuggy]

I am fully on board with this. :+1: May I suggest we add some sort of static analysis to our continuous integration pipeline, as well?

E.g., even in C++11, it is still possible to use language constructs that are not memory safe, b/c they do not do proper bounds checking. My understanding is that with C++ STL containers, using the [] array indexing operators is not bounds-checked. Using slice has the same problem. However, using the at() syntax does do proper bounds-checking, with minimal overhead, and the same functionality. So we'll probably want to get in the habit of using at() over the other forms. Maybe even set up some static analysis rules to catch this.

Trying to remember where I read all these pieces of information. I'll provide links later, if I can. Feel free to verify them for yourselves, i.e. in the c++11 official docs.

[stephengtuggy]

Also: Are we committed to using Travis CI as our Continuous Integration platform? The GitHub community as a whole seems to have migrated over to CircleCI. And I have quite a bit of experience with CircleCI, and would be happy to provide assistance with the migration, as I have time.

Up to you guys though.

[BenjamenMeyer]

@stephengtuggy I don't think we're tied to any CI system yet. Before we do any migration let's get the last of the Travis-CI PRs merged.

[stephengtuggy]

@stephengtuggy I don't think we're tied to any CI system yet. Before we do any migration let's get the last of the Travis-CI PRs merged.

@BenjamenMeyer Sure thing.

[BenjamenMeyer]

Since we use Boost in right now we probably should favor Boost::Test; that said, if something else is a better fit then great - if anyone wants to drop some alternatives here we can do a quick evaluation of them.

[stephengtuggy]

There's also CTest, which apparently integrates with CMake or is part of it (much like CPack).

[BenjamenMeyer]

So I use CTest and Boost::Test in another project I have - see https://github.com/BenjamenMeyer/samplingEngine/tree/master/src.

Essentially, Boost::Test implements the tests while CTest orchestrates the tests. Much like CPack orchestrates the packing tools to make RPMs/Debs, builds NSIS projects, and create Mac installers. You can check out https://github.com/BenjamenMeyer/samplingEngine/ for how it all works together.

NOTE: It's been a while since I've worked on that project; so it might be a little out of date build wise, but you should probably be able to do:

$ mkdir build
$ cd build
$ cmake ../src
$ make
$ make test

[stephengtuggy]

So we have unit tests now! Can this issue be closed?