search

keenlabs / keen-sdk-net

A .NET SDK for the Keen IO API
MIT License
37 stars 23 forks source link

AppVeyor CI + Coveralls #94

Closed masojus closed 7 years ago

masojus commented 7 years ago

Put in place some config to start building pushes/PRs with AppVeyor and capturing Code Coverage numbers, reporting them to Coveralls.io and updating badges in the readme file.

Eventually this should all be linked to the keenlabs GitHub org, but we can figure that out later once we've baked the whole process a bit more. Here are some examples of the test results, code coverage reporting in Coveralls, github checks, jobs running in AppVeyor, and updated badges:

badges code_coverage github_checks multiple_jobs test_results

masojus commented 7 years ago

Note that I'll turn on secret var decryption for PR builds for PRs coming from a branch in this repo, assuming only trusted users (contributors with write access) can create those. AppVeyor won't decode secret vars for PRs coming from forks, so Coveralls reporting wouldn't work there, which might cause the build to fail even though tests passed. Not having cc reporting for those builds is one thing, but failing CI because of it is problematic.

A few options might include squelching the return value from the coveralls.net reporting tool and/or making coverage run in a different job by defining a matrix using the env vars dimension or something, then skipping all but one job for coverage (no sense reporting cc for both VS 2015 and 2017), and using AppVeyor's matrix.allow_failures config to let the code coverage job fail without failing the build.

masojus commented 7 years ago

Another option might be to not embed the coveralls token in the .yml at all, since I think AppVeyor will merge env vars from the UI config with env vars in the .yml file.

masojus commented 7 years ago

It should also be easy enough to pivot on one or more of the env vars AppVeyor sets, like APPVEYOR_BUILD_WORKER_IMAGE to decide whether to upload coverage numbers, if we want.

masojus commented 7 years ago

It looks like AppVeyor UI vars override vars in the .yml file so it doesn't hurt to have the secure token in there, but this env var merging surely can't be allowed for vars declared as secure since that wouldn't make sense. The solution in order to allow PRs from all forks to automatically upload code coverage is to completely define the config via the UI, but then changes to it aren't tracked in the repo. Otherwise all forks have to target a topic branch in the repo, then contributors with write access have to re-PR to merge to protected branches.