Continuous integration

[masojus]

Get some or all tests unit tests running. Eventually if we also have end-to-end or integration tests that aren't appropriate for CI, we'll want to make sure those aren't included here.

Step 1 is going to be choosing a CI provider. Other Keen SDKs tend to use Travis-CI so we could look there first and create an account with all the correct permissions in place and such.

Step 2 is put in place whatever .csproj/.sln and/or Travis-specific config files (like .travis.yml) are necessary.

Then we need to address related issues around expanding/improving tests and getting code coverage numbers being recorded.

Lastly we'll want this stuff reported via widgets in the readme.

[masojus]

https://docs.travis-ci.com/user/languages/csharp/

Note that it suggests that they only support this via .NET Core, so there's that. That might mean using a different provider or only testing the netstandard binary this way?

[masojus]

We'll need to take into account which CI providers have integrations with Code Coverage reporting tools.

Some CI providers for .NET:

• AppVeyor - Free for Open Source projects
• Jenkins - Not sure
• TeamCity - Not sure of price for .NET
• VSTS - Some features free for small teams, not sure about Open Source for build/release/test features
• Travis CI - caveats listed above, namely only uses the .NET Core tools
• CircleCI - lists C# as a language, but there's no language guide for it, so not sure??
• CruiseControl.NET??
• Bamboo??

But then there are reporting/gathering tools like Coveralls, OpenCover, NCover, MSTest's own coverage tools, dotCover from JetBrains which isn't free (I think). There are projects like coveralls-net that do uploading to Coveralls.io for .NET, but that particular tool only currently supports AppVeyor and Jenkins maybe? NCover has integrations with Jenkins, TeamCity, Bamboo, CruiseControl, Code Central, and others.

Some of these CI providers also do deployment steps, like to Azure, NuGet, Github releases, AWS, etc. They also aren't mutually exclusive--AppVeyor can set up webhooks to watch branchs in Github, Bitbucket and VSTS, for example, and reporting tools can supposedly upload coverage results to Coveralls.io (or something else) from builds in a bunch of those providers. Some of these report results back in some form or other that's easy to embed as a widget in the readme. Different subsets of these tools support different subsets of test runners/frameworks (MSTest, xUnit, NUnit, ...).

So, we need to pick the right combination to support this Issue as well as Issue #72 . Just getting Unit tests running on changes from a given set of branches should be pretty easy in AppVeyor or VSTS, but may or may not solve the grander issue.

[masojus]

This article is from early last year, but does describe an end-to-end CI+CC scenario, using AppVeyor as the build server, nUnit as the test runner, OpenCover to gather coverage, coveralls.net to upload results to Coveralls.io, and badges in Github to report results. So, that may be a pretty solid first round of getting some of this stuff in place.

Here is another example that uses a similar set of tools.

[masojus]

Here's a recent article using OpenCover and reporting to Coveralls.io for a .NET Core project. So that's good news too hopefully for the move to a netstandard2.0 binary instead of the PCL.

[masojus]

This is basically done as part of the work in PRs #124, #125, and #126