Continuous integration

[zeehio]

I would like to add to mimic continuous integration infrastructure.

I would like on every commit to:

• Know if all the tests pass
• Know the percentage of code covered by tests
• Have a static code analysis report looking for potential issues
• Have a report of other checks (code style...)

To achieve that I plan to:

• Use Travis CI (To build the project)
• Use coverity for static code analysis
• Use code climate for the rest

I have some experience with these tools but before implementing this I would like to know:

• If any of you know of better or other tools
• If MycroftAI as an organization is happy to have that setup
• Your opinion, comments and feedback

[ryanleesipes]

Sure. We have a TravisCI account. I'll set that up and hook you up.

[forslund]

I am only familiar with coveralls but it doesn't seem to have as many features as coverity.

[zeehio]

Coverity does static code analysis while coveralls report code coverage. They are complementary I think. For C code I used coveralls in the past (I had forgotten its name, thanks @forslund)

So setting up Travis + coveralls + Coverity will be my first approach to deal with this.

[forslund]

Sounds like a plan. We might have to use a different test-library if we want to have automated coverage testing. cutest isn't ideal if we want a single unittest executable (which I believe would be good for gcov output).

[zeehio]

I believe a single executable is not needed. I don't recall having problems with gcov and multiple executables in the past (my Travis + coveralls + coverity example is here)

[forslund]

It's always great to be wrong =)

I've only used gcov-coverage with a small vim-plugin and as I recall the gcov output files was overwritten each time the software was started. Travis + coveralls + coverity are probably smarter than my little tool.

[LongBoolean]

I would encourage the use of asserts. Along with the mindset of, If an assert fires off, that is a bug, stop everything and fix it. If a particular assert was written for any other purpose than to catch a bug then it should not be an assert. (don't use asserts as if they are a message)

[LongBoolean]

Question: What does mimic currently use for unit testing? (coming over from flite)

[forslund]

I rewrote the tests for flite that looked roughly like unittests using the minimal header-file-only cutest library. Mainly because it was simple and didn't require any extra libs.

[rhdunn]

Asserts in library code are considered bad practice. This is because they can easily take down the application/daemon using them (e.g. an assistive technology application such as a screen reader), and provide little feedback of the problem to the user/application.

[forslund]

Partially implemented in #27. Static code analysis remains to be completed.

[forslund]

Merge of PR #29 resolves this issue. Closing